VIP 27: Move some VCL variables to the catflap facility

Synopsis

VCL currently has two "lookup modifiers" with the variables req.hash_always_miss and req.hash_ignore_busy. The goal behind this VIP is to evaluate the relevance of turning them into catflap consumers.

Why?

The VCF facility is currently undocumented on purpose and only has test coverage via vmod-debug. Turning those two variables into catflap functions would remove special cases from the core lookup code and help work out actual needs when it comes to tweaking cache lookups and how to compose tweaks. Shipping catflaps in tree might also encourage third-party catflaps once we reach a stabilization point.

How?

There is no clear "how" currently. The existing 4.0 and 4.1 VCL syntax will need to maintain the existing variables, but we could imagine shipping a vmod-lookup or something similar to offer built-in VCFs.

In order to compose VCFs, we need to update the current API so that more than one can be registered. We might consider a registration similar to beresp.filters:

sub vcl_recv {
    if (client.ip ~ varnish_neighbors) {
        set req.lookup = "ignore_busy";
    } else if (req.method == "REFRESH") {
        set req.method = "GET";
        set req.lookup = "always_miss";
    }
}

@nigoroll has another "prior art" proposal for this and will add it to this document.

A new lookup modifier can be considered too: some kind of hash_ignore_bgfetch that ignores a busy object only when an eligible stale object is also available.

sub vcl_recv {
    if (req.http.host ~ "very slow backend") {
        set req.lookup = "ignore_bgfetch";
    }
}