VIP 26: limit private prefetch

Synopsis

• Add means to limit the amount of data prefetched for private objects
• Limit the Transient storage in size by default

Why

While HTTP/1 pipe mode uses a fixed buffer to relay the backend response, for private objects via pass/hfm/hfp, all backend response data is prefetched as quickly as possible from the backend. This requires potentially unbound amounts of storage and may trigger unexpected errors when transient stevedore allocations (malloc() in most practical cases) fail. User expectations are that private object require virtually no cache memmory.

Ref: https://github.com/varnishcache/varnish-cache/issues/2964

How

• Make the Transient Stevedore limited to 100MB by default
• add a parameter pass_prefetch with 1MB default
• make pass_prefetch adjustable from vcl as bereq.pass_prefetch
• add a no-fail mode to stevedore allocations where the requester gets notified (via a CV) by the stevedore as soon as there is memory available again.
  • add a counter for "waiting for storage" events
• use the no-fail mode for private objects

Parameter considerations

The idea of a pass_maxchunksize parameter came up during discussions because the current default value for fetch_maxchunksize (0.25G) would make this feature useless out of the box. It shouldn't be needed if pass_prefetch takes precedence.

Workaround

A workaround by using pipe method has been discovered, by using the content length header in the backend response. If (in this example) it shows that the response exceeds 30GB then (by chaining through several methods) restart the request and go into pipe mode.

vcl 4.1;

import std;

sub vcl_backend_response {
    if (std.integer(beresp.http.content-length, 0) > 32212254720) {
        std.log("DEBUG: Fail this error request, as Transient storage might be insufficient.");
        set bereq.http.x-error-reason = "oversize";
        return (error);
    }
}

sub vcl_backend_error {
    if (bereq.http.x-error-reason == "oversize") {
        set beresp.status = 599;
        return (deliver);
    }
}

sub vcl_deliver {
    if (resp.status == 599) {
        std.log("DEBUG: restarting, backend fetch failed due to oversize");
        set req.http.x-pipe-request = "1";
        return (restart);
    }
}

sub vcl_recv {
    # If we have a restart from vcl_deliver, then we immediately go into pipe
    if (req.http.x-pipe-request && req.restarts > 0) {
        return(pipe);
    }
}