Stale-while-revalidate header unexpected behavior

Support
, ,
1

TLDR in comment

Hi,

I have an Astro project in SSR mode which uses the netlify adapter. It seems to fire the entry lambda function every time a request is made to the site. This function fetches a Wordpress API for every page request that takes about 3 seconds to resolve. It then builds the requested page based on the resolved data.

Expected behavior:
I’m trying to make use of the Netlify stale-while-revalidate header to serve a stale version of the page and then request the cache to fill itself with fresh content and serve that on the following request.

Current behavior:
It works for about 15 to 30 seconds (varying results). And then starts fetching the fresh content again for every page, even though it should still be stale as I’m setting it to return stale content for 7 days.

The header I’m using:

Netlify-CDN-Cache-Control: public, max-age=0, stale-while-revalidate=604800

What I have tried:

  • Using a middleware.ts file where I fetch the data in an onRequest function and then pass it to Astro.locals.pageData and setting the
    Netlify-CDN-Cache-Control: public, max-age=0, stale-while-revalidate=604800
    on the fetch’s headers instead of the page’s. This seems to crash the build on Netlify.

  • I have tried setting a _headers and netlify.toml file in conjunction with having the header set in a […slug].astro file, even though this is not supported for SSR.

  • I have tried setting the Astro response headers in the […slug].astro file, which handles every path. This seems to be the way it’s supposed to work? But it stop serving stale content after 15-30 seconds (varying).

  • I have tried fiddling with the max-age of the header, setting max-age=1 for example instead of 0. And settings s-max-age=604800 (which shouldn’t matter).

This is the minimal reproduction:
https://fascinating-otter-d9ae75.netlify.app/

And the Github repo:
https://github.com/wsbos/united-disk

Any help or advice would be appreciated :slightly_smiling_face:

2

Update:

This example by Netlify has the same unexpected behavior:

Repo:
https://github.com/netlify-labs/cache-control-and-swr

Demo site:
https://netlify-edge-cache-demo.netlify.app/swr

If you open up the network tab in your console you can see the initial request taking about 400ms, then subsequent requests take about 20ms. Then when you wait for 60 seconds and refresh it, it takes 400ms again. But it should actually serve the last background request you made before this, so the one with an age of 59.6 seconds. (60 - the time the background request took). But it actually serves one with an age of 0 or 1. I’ve also noticed that when this happens, the Remote Address was changed. Sometimes I land on a new Remote Address and it has an age of far more seconds than I waited.

3

This is very likely related to: Cannot get Netlify api endpoint caching to work as expected - #7 by hrishikesh

4

Yes it’s the same kind of issue. But if it serves the site from a different CDN node every 60 seconds, even on the same device in the same location, this is not a viable way of doing stale-while-revalidate. It would only really be viable for websites with a load of traffic filling all the CDN nodes.