Result of builder not served compressed for gltf data (no content-encoding header)

I’m currently working on an application to retrieve 3D data from an external service (Onshape) and cache it in the Netlify Edge using a builder to enable embedding the 3D data in external websites with great performance.

However, it seems that Netlify is serving the content without compression, resulting in 1 MB of data transfer in a case where brotli compression results in less than 200 KB file size. This size conclusion is based on using https://www.multiutil.com/text-to-brotli-compress/ to compress the file which is served for the following website (I can’t attach it directly due to being a new user in the forum):

Website: Onshape Model Viewer

Specific builder request URL that’s not compressed: https://embed-onshape.netlify.app/edgecache/d/177a60f6104ae7febf4393b6/v/7262d5efaab7231440d91a24/e/460fabf8c7fbf3104102a870/psgltf

The repo is not public at the moment, but I can see if I can get permission to post it from work. (I’m currently using my personal account to hack together a quick prototype/proof of concept, will make a work account very soon :smiley:)

General request info:

Request URL: https://embed-onshape.netlify.app/edgecache/d/177a60f6104ae7febf4393b6/v/7262d5efaab7231440d91a24/e/460fabf8c7fbf3104102a870/psgltf
Request Method: GET
Status Code: 200 OK
Remote Address: 34.168.247.115:443
Referrer Policy: strict-origin-when-cross-origin

Request headers (note accept-encoding: gzip, deflate, br - I know this should be br, gzip, deflate for best performance and I’ll fix that, though right now no compression is used):

GET /edgecache/d/177a60f6104ae7febf4393b6/v/7262d5efaab7231440d91a24/e/460fabf8c7fbf3104102a870/psgltf HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,ja;q=0.8,lv;q=0.7
Connection: keep-alive
DNT: 1
Host: embed-onshape.netlify.app
Referer: https://embed-onshape.netlify.app/?did=177a60f6104ae7febf4393b6&wvm=v&wvmid=7262d5efaab7231440d91a24&eid=460fabf8c7fbf3104102a870
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36
sec-ch-ua: "Google Chrome";v="111", "Not(A:Brand";v="8", "Chromium";v="111"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"

Response headers (note lack of content-encoding header):

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Age: 128293
Cache-Control: public, max-age=0, must-revalidate
Content-Length: 1041326
Content-Type: model/gltf-binary
Date: Mon, 27 Mar 2023 07:52:53 GMT
Server: Netlify
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-Nf-Request-Id: 01GWMV7QXTRPP0BYK2C6Y90G8E

For debugging steps, I’ve checked out some documentation and forum posts, and they seem to indicate that things should automatically be compressed if they are larger than 1 KB and there is a benefit from compression (both are true in this case - 1 MB file to 0.2 MB compressed with brotli, significant benefit from other compression formats too).

Please let me know if there is anything else I can do to debug this, and I’m very sorry if I’m missing something obvious!

Ultimately, I realize it would probably be best to store a compressed result from the builder (particularly since larger models could exceed the 6 MB limit, but assuming similar compression performance, it might be possible to 5x the “actual size” of the model that can be stored).

Thank you in advance for any help and ideas!

Hi @alnis :wave:t6: welcome back to the Netlify support forums. I’m glad you found us.

Have you tried inspecting the response headers on your browser and verify that the “content-encoding” header is missing? If the header is present, it means that compression is being applied, and the issue lies elsewhere.

Netlify does automatically compress content larger than 1KB. Could you try setting a lower size threshold in your config and see if that helps?

Hi @SamO, thank you for the reply!

The request headers include Accept-Encoding: gzip, deflate, br but the response headers do not have a Content-Encoding header.

Sorry, how would I change the size threshold in the config? I’m having trouble finding documentation on this. Note that the Content-Length: 1041326 header implies that the content is 1.041 MB which should be well over the 1 KB minimum.

Screenshot of headers:

Thank you very much for your time and help.

I found this on google…

[[compress]]
path = “/*”
threshold = xyz

You would add this to your netlify.toml file.

Hello @SamO,

Sorry for the long delay! I tried that out, and while I could get Netlify to compress even small static files, there was no difference for builder results. They were still served without compression.

I wonder why serve the model from an ODB instead of a static file directly?

The contents are not known at build time, since the models are determined by end users.

The flow of data is:

  1. End users create a URL for this app based on the Onshape Document, Version, and Element IDs
  2. When loading for the first time, the ODB fetches the graphics data from Onshape, but since it is an ODB, the result is cached on the Netlify edge
  3. Future loads of the page are accelerated due to the caching of the result of the ODB in Netlify edge, also preventing the need to hit the Onshape API on each load

Note that the document/version/element IDs are determined by end users, and more are added over time as additional people use the application, so it isn’t possible to precompute at build time.

While I could probably rig something up using a separate CDN from Netlify or maybe even just caching everything in S3 (or similiar) resulting in worse performance, I was hoping to keep everything inside the Netlify ecosystem by using ODBs :smile:

I understand this might be an unconventional use case, so thank you for your patience with this!

Hi, @alnis. Netlify only automatically compresses responses for static assets. Your On-demand Builder response is a serverless function and the automatic compression isn’t enabled there.

If you want to returned compressed content from a function, there are some examples in this StackOverflow post:

If there are other questions about this, please let us know.