Excessive Bandwidth Usage

Hi, @nraboy. I also had the same question when I was first learning about how our caching works when I experienced the same issue. There are both a reason and solution for this behavior!

The -df on the end of the etag (1be33a500fe9965531b55b00eb157868-ssl-df) indicates that this is the etag for the compressed version of the URL (df = deflate).

There was a 200 status response because curl normally requests the uncompressed version of a URL so the etag isn’t actually a match. However, you can add an accept-encoding request header and the 304 will occur like so:

$ curl -I -H 'If-None-Match: "1be33a500fe9965531b55b00eb157868-ssl-df"' -H 'accept-encoding: gzip, deflate, br' https://www.thepolyglotdeveloper.com/css/bootstrap/bootstrap.min.css
HTTP/2 304
date: Tue, 03 Sep 2019 02:15:08 GMT
etag: "1be33a500fe9965531b55b00eb157868-ssl-df"
cache-control: public, max-age=0, must-revalidate
server: Netlify
vary: Accept-Encoding
x-nf-request-id: 56251d0c-bbc9-4172-aa60-6a5e55b7c899-3247103

Adding this additional header (using -H 'accept-encoding: gzip, deflate, br' ) means curl is now requesting the gzipped version of the page and this will cause the -df etag header to match - resulting in 304s.

Most web browsers always include the accept-encoding: gzip, deflate, br header by default. On the other hand, curl only sends that header when it is specifically told to do so.

You can also used --compressed in place of that header:

curl --compressed -I -H 'If-None-Match: "1be33a500fe9965531b55b00eb157868-ssl-df"' https://www.thepolyglotdeveloper.com/css/bootstrap/bootstrap.min.css

Additional questions are always welcome. :+1:

1 Like