Image CDN is broken

The image CDN seems to be broken and returning 404 most of the time. The following curl command will return the correct 200 and image maybe 1 out of 10 requests.

curl 'https://developers.poki.com/.netlify/images/?url=/images/defold.png&w=500&fm=webp&q=75' \
  -v \
  --compressed \
 -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:125.0) Gecko/20100101 Firefox/125.0' \
 -H 'Accept: image/avif,image/webp,*/*' \
 -H 'Accept-Language: en-US,en;q=0.5' \
 -H 'Accept-Encoding: gzip, deflate, br' \
 -H 'Referer: https://developers.poki.com/' \
 -H 'Pragma: no-cache' \
 -H 'Cache-Control: no-cache' > /dev/null

This is what a correct response looks like:

< HTTP/2 200 
< accept-ranges: bytes
< access-control-allow-origin: *
< age: 72731
< cache-control: public,max-age=86400,stale-while-revalidate=86400,stale-if-error=86400
< cache-status: "Netlify Edge"; hit; fwd=stale
< content-security-policy: frame-ancestors 'self'
< content-type: image/webp
< cross-origin-resource-policy: cross-origin
< date: Wed, 10 Apr 2024 11:07:05 GMT
< last-modified: Tue, 09 Apr 2024 10:02:54 GMT
< netlify-vary: query=url|crop|fit|fm|h|height|position|q|quality|timestamp|w|width
< server: Netlify
< strict-transport-security: max-age=31536000
< timing-allow-origin: *
< vary: Accept
< x-content-type-options: nosniff
< x-imgix-origin-passthrough: Cache-Control=public%2Cmax-age%3D86400%2Cstale-while-revalidate%3D86400%2Cstale-if-error%3D86400&Content-Security-Policy=frame-ancestors+%27self%27&X-Nf-Cache-Control=
< x-nf-request-id: 01HV3V18VE9N1C1VPQ51ZNRTY3
< content-length: 10314
< 
{ [4096 bytes data]

This is the bad response I’m getting most of the time:

< HTTP/2 404 
< age: 52097
< cache-control: public,max-age=0,must-revalidate
< cache-status: "Netlify Edge"; hit
< content-encoding: br
< content-security-policy: frame-ancestors 'self'
< content-security-policy-report-only: default-src 'self' *.poki.io *.poki.com *.googleusercontent.com *.gravatar.com *.mxpnl.com api-js.mixpanel.com cdn.jsdelivr.net data: 'unsafe-inline' https://storage.googleapis.com/poki-playtest-recordings/; object-src 'self' *.poki-gdn.com; frame-ancestors 'none'; form-action 'self' devs-api.poki.io; upgrade-insecure-requests; block-all-mixed-content; disown-opener; report-uri https://t.poki.io/csp
< content-type: text/html; charset=utf-8
< date: Wed, 10 Apr 2024 11:09:05 GMT
< etag: 1667901135-ssl-df
< server: Netlify
< strict-transport-security: max-age=31536000
< vary: Accept-Encoding
< x-frame-options: DENY
< x-nf-request-id: 01HV3V4XDT6XWRKK4RFEG1R2S9
< x-xss-protection: 1; mode=block
< content-length: 1298
< 
{ [1298 bytes data]

The issue seems to have been fixed now. But this was going on for hours. It’s kinda sad that https://www.netlifystatus.com/ was never updated to reflect this issue.

This happed to be a bug which was noticed and fixed shortly after.

But the image CDN was down for at least half a day. Shouldn’t that be mentioned in https://www.netlifystatus.com/ ? Otherwise what is that status page for?

We’ve just updated the status page to declare this incident retroactively: https://www.netlifystatus.com/

Thanks for calling that out!