Varnish + requests with no Content-Length header

HTTP 1.1 introduced the concept of chunked transfer encoding. This (among other things) enables us to send a request without knowing how large the content is going to be at the time we start the request.

An example usage would be where you generate content on-the-fly. You could potentially send chunks of a video-stream while you are recording or dynamically alter (compress, parse or similar) content from one source and incrementally send chunks to the server.

In node.js, if you send an HTTP request without a Content-Length header, node will automatically add Transfer-Encoding: chunked to the list of headers, as well as automatically split your data into chunks.

Varnish does not currently treat requests without a Content-Length header very well. It adds a Content-Length header with a value of 0 and sends it to your backend without any data, which isn’t what we want. To fix this, you can add a simple check in your VCL to instead pipe the request directly to your backend:

if ((req.request == "POST" || req.request == "PUT") &&
    req.http.transfer-encoding ~ "chunked") {
    return(pipe);
}

Note: The way Varnish handles chunked transfer encoding might change in the future.

Enjoy streaming!