How to cache on-the-fly generated images in the CDN?

Support
,
1

I’m building a Remix blog which I’ve deployed to Netlify. So far so god. Now I want to transform images with sharp from an API route and cache the result in the CDN. The route (/api/image?...) returns the image and the cache-control header 'public, max-age=31536000, s-maxage=31536000', but I always get age=0 in my response headers. I assume that this means that the image gets generated over and over again? If I set s-maxage, shouldn’t this get cached in the CDN? Is there any other option to check for cache hits beside the age response header?

image

2

Hi @peter.oesteritz

Could we have a site to check? We have the request ID in the screenshot, but typing that from the image is error prone. So, it would be okay if you can send us the text of that.

Furthermore, age: 0 would mean it was not cached in the CDN node that served it. It could be cached in one of the 70+ other nodes.

3

Hi @hrishikesh, here’s the link: https://deploy-preview-6--debd-com-2022.netlify.app/cases/discover

image
image3584×2080 1.03 MB

4

It seems to me you’re using a normal Function. Did you try using an On-Demand Builders: On-demand Builders | Netlify Docs?

5

Hi @hrishikesh, no I didn’t. I’ve just created a route in my Remix app. It this necessary? Because I want to create a lot of other routes in my Remix app too (static content routes) which I would like to cache for at least 60 seconds. If I can’t set s-maxage on Remix routes, Remix@Netfliy wouldn’t make much sense :frowning: (Remix | Conventions)

7

Hey there, @peter.oesteritz :wave:

Thanks for your patience here, we appreciate it. Are you still encountering obstacles? If so, can you please share your repo with us?

If you cannot share your entire repo, please share your netlify.toml file and how you have defined your headers.

8

Hi @hillary,

unfortunately I can’t share the repo, but here’s the netlify.toml:

[build]
  command = "npm run build"
  publish = "public"

[dev]
  command = "remix watch"
  port = 3000

[functions]
  included_files = ["app/content/**", "content/**", "public/build/_assets/**"]

[[redirects]]
  from = "/*"
  to = "/.netlify/functions/server"
  status = 200

[[headers]]
  for = "/build/*"
  [headers.values]
    "Cache-Control" = "public, max-age=31536000, s-maxage=31536000"

And here’s my Remix loader which sets the headers for the images (you can see the header that I set for the image response in the screenshot from my answer above):

export const loader: LoaderFunction = async ({ request }) => {
  const image = imageLoader(config, request)

  return new Promise((resolve, reject) => {
    image
      .then((response) => {
        response.headers.set('Cache-Control', 'public, max-age=31536000, s-maxage=31536000')
        resolve(response)
      })
      .catch((reason) => reject(reason))
  })
}
9

Hi, @peter.oesteritz. I do show those headers being returned for URLs backed by the function. For example:

$ curl --compressed -svo /dev/null 'https://deploy-preview-6--debd-com-2022.netlify.app/api/image?src=%2Fbuild%2F_assets%2Ftest-OUV23Z46.png&width=101'  2>&1 | egrep '^(<|>)'
> GET /api/image?src=%2Fbuild%2F_assets%2Ftest-OUV23Z46.png&width=101 HTTP/2
> Host: deploy-preview-6--debd-com-2022.netlify.app
> user-agent: curl/7.79.1
> accept: */*
> accept-encoding: deflate, gzip
>
< HTTP/2 200
< age: 0
< cache-control: public, max-age=31536000, s-maxage=31536000
< content-type: image/png
< date: Thu, 16 Jun 2022 07:44:52 GMT
< server: Netlify
< x-nf-request-id: 01G5NQHQB70EHD6RCZHXP1EGVS
<

That does show the header cache-control: public, max-age=31536000, s-maxage=31536000 sent and I confirmed that the function is what generated the response.

Please note, the Standard Edge Network is shared. You will never have direct control of the cache there. Even if you use these cache headers, unless your site has very high traffic, the cached URLs are likely to drop out of cache quickly.

Looking at the site in question, there are only 10 request being served by this function in the last 24 hours. Based on that very low traffic level, the URLs will be quickly be ejected from the cache.

This is because this header below is about optional caching only:

cache-control: public, max-age=31536000, s-maxage=31536000

Says that a CDN can optionally cache this URL for 365 days. It does not say the CDN must cache this URL. It says a CDN can cache it if it wants to. Again, as the Standard Edge Network is a shared resource with millions of Starter plan sites on it, low traffic sites will always be quickly ejected from the cache.

So, you are applying the headers correctly. However, the headers are also not causing the images to be cached because the site is low traffic and you are on a shared CDN.

The On-demand Builder functions do allow for caching of the response, even for low traffic sites. The caching is done at the origin (not in the CDN) so a single cached response does persist for all CDN nodes, again, even if the traffic is low.