Headers in TOML not applying

Support
1

Site name: thunderous-bubblegum-8bc069

I’m having trouble getting headers in my netlify.toml to apply. My configuration:

# netlify.toml
[build]
  base = "apps/www/"

# apps/www/netlify.toml
[build]
  command = "pnpm run build"
  publish = "dist"

[[headers]]
  for = "/_astro/*.:fingerprint.:ext"
  [headers.values]
  cache-control = '''
  public,
  max-age=86400,
  stale-while-revalidate=2592000,
  stale-if-error=2592000,
  immutable'''

Failing test case:

curl -svo /dev/null "https://deploy-preview-19--thunderous-bubblegum-8bc069.netlify.app/_astro/index.126e85d8.css" 2>&1 | grep cache-control
< cache-control: public, max-age=0, must-revalidate

My goal is to match everything like /_astro/confident-crow.3c210aae.svg and /_astro/james-pixelated.2abe0d5f.png. It doesn’t seem to be matching anything.

Are multiple :placeholders supported? If not, how else would you express something that matches these fingerprinted files, but doesn’t match something like /_astro/foo.bar/baz.js?

If it were possible to use a regular expression, I would use

for = "/_astro/.+\.[a-f0-9]{8}\.[a-zA-Z0-9]+"

to be as precise as possible.

2

Hi @jamesarosen ?

Welcome to the forums and thanks so much for reaching out. Where is your netlify.toml? Is this in the root directory of your project and located in the same package.json otherwise it won’t be recognized by our system. If this doesn’t help can you please give a read of our documentation? Custom headers | Netlify Docs

3

I use a monorepo, so I have two netlify.toml files and two package.json files:

# netlify.toml
[build]
  base = "apps/www/"

// package.json
{ name: "@my/workspace" }

# apps/www/netlify.toml
[[headers]]
…

// apps/www/package.json
{ name: "@my/www" }

If I define the headers the workspace root, then I’ll get the same values for every application, which is undesirable.

The docs do say

When you declare headers in a _headers file stored in the publish directory or a Netlify configuration file, the headers are global for all builds and cannot be scoped for specific branches or deploy contexts.

But this isn’t about branches or deploy contexts.

Edit
I tried moving the [[headers]] to the root netlify.toml in preview build 22, but they’re still not working.

4

I tried moving the config to a _headers file in the workspace root:

/_astro/*.:fingerprint.:ext
  cache-control: public
  cache-control: max-age=86400
  cache-control: stale-while-revalidate=2592000
  cache-control: stale-if-error=2592000
  cache-control: immutable

That didn’t work. Nor did that same file in apps/www/_headers.

I also figured that Netlify might have a problem with *.:fingerprint.:ext (despite there being no limitation listed in the docs), so I tried this in both locations:

/_astro/*.*
  cache-control: public
  cache-control: max-age=86400
  cache-control: stale-while-revalidate=2592000
  cache-control: stale-if-error=2592000
  cache-control: immutable

Neither worked. At this point, I’ve tried every option that the Headers doc suggests might work.

5

It turns out the problem is that the @astrojs/netlify package copies _redirects, but not _headers to the dist/ directory. Manually copying _headers fixes the issue. I’ve opened an issue on that project: [Netlify] poor support for Netlify's headers config · Issue #6510 · withastro/astro · GitHub

Though I do want to point out that even now that Netlify recognizes the configuration, the matcher /_astro/*.:fingerprint.:ext doesn’t match anything.

If a matcher can only have a single :placeholder, please document that limitation. The docs currently say,

  • Paths can contain * or :placeholders. A :placeholder matches anything except /, while a * matches anything.

:placeholders suggests to me that there can be any number.

6

Your comment on that issue seems like an assertion, and an incorrect one. Multiple placeholders work perfectly fine. I’d recommend deleting (or editing) the comment from that issue to avoid any confusion. Check out the repo:

and the site:

https://forums-87328.netlify.app/nested/index.html

Header is applied:

image

The issue in your example is that you’re using *. When we see an asterisk, that triggers a wildcard match, so everything after * is ignored.

7

Fascinating. That’s definitely not clear to me from the docs.

I forked of your app and ran some further tests:

This rule

/*/:file.js
  is-js: true

applies to /nested/index.html, though I wouldn’t expect it to.

curl -svo /dev/null https://frabjous-halva-7b75f1.netlify.app/nested/index.html 2>&1 | grep js
< is-js: true

That suggests that */ causes the rest of the path to be completely ignored.

This rule

/test/*.:fingerprint.css
  is-test: true
  fingerprinted: true
  star-count: 1
  placeholder-count: 2
  slash-count: 1

doesn’t apply to /test/foo.abc123.txt, though I would expect it to.

curl -svo /dev/null https://frabjous-halva-7b75f1.netlify.app/test/foo.abc123.txt 2>&1 | grep is-test
(empty)

That suggests that * alone doesn’t cause the rest of the path to be completely ignored.

Most router frameworks I’ve worked with (Rails, Ember, React-Router, Astro, Jekyll) have these same concepts of “glob” and “placeholder.” They all have the same contract. They turn a * (glob) into a .+ RegExp and a :placeholder into a [^\/]+ RegExp. Thus, I would expect /*/:file.js to match the same things as /^\/.+\/[^\/]+\.js$/, and /test/*.:fingerprint.css to match the same things as /^\/test\/.+\.[^\/]+\.css$/.

8

I’m seeing all your headers on https://frabjous-halva-7b75f1.netlify.app/test/foo.abc123.txt:

image

I don’t see that happening on Netlify. Our behaviour has been established that way for years, and changing that suddenly would mean a breaking change for not sure how many sites.

A simple rule like I mentioned before:

* will ignore everything after it, use placeholders if you don’t want wildcard

9

If that “rule” were correct, /test/*.:fingerprint.css would match the same things as /test/*, which it does not. https://frabjous-halva-7b75f1.netlify.app/test/foo.abc123.txt should have is-test: true from

/test/*.:fingerprint.css
  is-test: true
10

See also Cloudflare’s documentation for _headers, which specifies support for things like /*.jpg. This is also consistent with Unix glob (in semantics, but not syntax).

13

Hi @jamesarosen, I’m on the docs team at Netlify.

Thanks for raising this content gap to our attention. We’ve added some more information on the topic here: Custom headers | Netlify Docs

14

Thanks, @rstav. Those docs are much clearer now :+1: