Bundle file not served GZIP

[Tom_Rupsis]

I’m trying to figure out why my site (whitewater.dev) is not serving the GZIP file. According to the following post it should be happening automatically:

Gzip file serving

In my quest for a faster site I´m looking to serve gzipped files. So I guess I have two options. 1 gzip my files in the build process 2 have the server do the gzipping for me if I were to go for number 2 I{m I correct in thinking that I would have to add this to my headers configuration in the netlify.toml file? Does any one have any tips or previous experience with this? Any tips greatly appreciated

But it’s not. I see the accept-encoding: gzip, deflate, br in the Request Headers. But I do not see content-encoding: gzip in the Response Headers. I run the Lighthouse performance test in Chrome/Edge and I get a 0 score for performance because it’s pulling the regular index_bundle.js file instead of the index_bundle.js.gz. I’ve got webpack configured to produce the .gz file.

I switched to Netlify DNS thinking maybe that was affecting the response, but no change. I switched to Cloudflare DNS and the performance is much better but still content-encoding: br.

I have a separate Gatsby site on Netlify that sends x-content-encoding-over-network: gzip, but this site sends x-content-encoding-over-network:br. What am I missing?

[luke]

Hi, @Tom_Rupsis, I’m seeing gzipped content now but the response is coming from Cloudflare not Netlify which limits our ability to troubleshoot what is happening at Netlify:

$ curl -svo /dev/null --compressed https://whitewater.dev/  2>&1 | egrep -i '< (HTTP/|content-encoding|server)'
< HTTP/2 200
< server: cloudflare
< content-encoding: gzip

Note, if I force the domain name to connect to Netlify, I also get gzipped content:

$ curl -svo /dev/null --compressed https://whitewater.dev/  --resolve whitewater.dev:443:104.198.14.52 2>&1 | egrep -i '< (HTTP/|content-encoding|server)'
< HTTP/2 200
< content-encoding: gzip
< server: Netlify

Are you still seeing this issue? If so, what URL shows the missing gzip header?

[Tom_Rupsis]

I have reverted DNS back to Netlify. The whitewater.dev site is a very simple React site. When I run the build locally, it creates index_bundle.js (5.7 MB) and index_bundle.js.gz (1.7 MB). The 5.7MB file is being served and I’ve uploaded a screenshot of the request/response headers I see in Chrome DevTools.

[luke]

Hi, @Tom_Rupsis, here is what I found.

The response in the screenshot was compressed. This is shown with this header:

x-content-encoding-over-network: br

So, the javascript file (not the gzipped version) was served using brotli. I showing the response size (including headers) was 1741649 bytes. This is smaller (even with headers) than the the gzipped bundle which is 1811427 bytes.

Also note, you do not need to pre-compress your assets to have them served compress at Netlify. In fact, the only way to get the gzipped bundle in the deployment is to specifically request it. This site won’t ever use the gzipped bundles in the deploy (without specifying the .gz).

Here is a curl example piped to wc -c to get the byte count (please note this counting method does not include headers):

$ curl -s https://whitewater.dev/index_bundle.js.gz | wc -c
 1811427

This is smaller than our gzipped version (using an accept-encoding header without brotli support):

$ curl -s -H 'accept-encoding: gzip, deflate' https://whitewater.dev/index_bundle.js | wc -c
 1819759

So, our gzipped is used and is a less optimized compression of the file. However, both Chrome and Firefox will use brotli which is smaller still. (Edge and Safari probably will too but I didn’t test them.)

Again, though, the brotli version (this time without counting headers) is still the smallest:

$ curl -s -H 'accept-encoding: gzip, deflate, br' https://whitewater.dev/index_bundle.js | wc -c
 1741210

To clarify, our service won’t use the local gzipped bundles unless the URL requested includes that exact filename. By this I mean the only way to get your gzipped version is to request index_bundle.js.gz not index_bundle.js and the site doesn’t do this.

We will send gzipped or brotli compressed versions of the files automatically and there is no need to pre-compress during the build (because those files won’t get used in most scenarios).

If there are other questions about this, please let us know.

[Tom_Rupsis]

Thank you for the detail, Luke. I’m sorry for seeming so dense, but I see index_bundle.js as a 5.7 MB download in Chrome DevTools (screenshot attached). I also get an “Avoid enormous network payloads” recommendation from Lighthouse performance test that shows the index_bundle.js size is 5,593 KB (screenshot attached). Why does this information conflict with what you’re seeing?

[luke]

Hi, @Tom_Rupsis, my best guess is that it is showing the uncompressed size for simplicity. When I tested just now, this is what I see:

If there are other questions, please let us know.

[Tom_Rupsis]

Well I’m not sure what’s going on. I’ve tried the site on various browsers, computers, and networks. Sometimes it downloads the 1.7MB file and sometimes the 5.6MB file. I can’t find any common thread that explains which file is served. It is definitely not showing the uncompressed size for simplicity as the browser will show you both the “resource size” and the “transferred over network” size. It must be something with the way some browsers/computers make the request that prompts Netlify to send one or the other. But the request headers always look the same so I am completely clueless (it’s not the first time!).

[luke]

Hi, @Tom_Rupsis, if you send us the x-nf-request-id header for any uncompressed responses, we can research them to find out more about what is happening.

Also, if you would be willing to send us a HAR file capture of the issue, that would be helpful as well.

Any additional details are appreciated. If there are other questions, please let us know and we’ll be happy to answer.

[Tom_Rupsis]

This request was sent uncompressed (5.7MB “resource size” and “transferred over network”:
c69a3b52-55ad-4bdc-ae65-3fddee7673bb-5363649

This request from another PC on the same network was sent compressed (5.7MB “resource size” and 1.7MB “transferred over network”:
6d981e4f-93b1-408c-9043-c91fec97aca1-2300069

HAR files for both compressed and uncompressed requests are here:

whitewater.dev - har files.zip

Thank you!

[luke]

Hi, @Tom_Rupsis, I’ve check all four requests for the URL:

• https://whitewater.dev/index_bundle.js

All four are getting the compressed brotli format. If some tool is telling you that some of these requested are uncompressed it is a bug in the tool itself. We sent all four compressed.

Note, I’m seeing a different user-agent for the request you report as uncompressed. Again, all responses were served compress but the reported compressed user-agent is this:

Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36

The user-agent reported uncompressed (but it is really compressed) is this:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36 Edg/83.0.478.54

That is the tool/browser lying to you. Don’t believe it.

There are interesting differences in our response headers to the versions. These are the “reported uncompressed” response headers from the HAR file:

accept-ranges bytes
age 60864
cache-control public, max-age=0, must-revalidate
content-type application/javascript
date Mon, 22 Jun 2020 20:43:00 GMT
etag "1f6377ea62a71ea60cc6f10e0c64de6c-ssl-df"
server Netlify
status 200
strict-transport-security max-age=31536000
vary Accept-Encoding
x-content-encoding-over-network br
x-nf-request-id feb6caee-8f1d-49fd-9f40-173cef5c5295-13375013

Note the etag ends in -ssl-df. This means this response is a compressed version (the -df is always in the etag for a compressed response and never there for uncompressed). The -ssl means this was an SSL response.

I also checked the x-nf-request-id of feb6caee-8f1d-49fd-9f40-173cef5c5295-13375013 in our logs. It was Brotli compressed and the response size was 1741641 bytes.

Last but not least, the x-content-encoding-over-network value of br also validates that this was Brotli compressed content. The br stands for Brotli.

If you check the headers for the reported compressed (again, they were all compressed) the headers are different but show all the same details (if in a different way):

accept-ranges bytes
age 348514
cache-control public, max-age=0, must-revalidate
content-encoding br
content-length 1741198
content-type application/javascript
date Fri, 19 Jun 2020 12:48:07 GMT
etag "1f6377ea62a71ea60cc6f10e0c64de6c-ssl-df"
server Netlify
status 200
strict-transport-security max-age=31536000
vary Accept-Encoding
x-nf-request-id 6d981e4f-93b1-408c-9043-c91fec97aca1-2309947

Again the etag ends with -ssl-df and the content-encoding is br which is Brotli. The transfer size in the logs was 1741641 bytes.

To summarize, yes, there is a bug but it isn’t at Netlify. It is in the browser or tool you are using. It might help to file a bug report with them about that.