_headers file rules not applying to custom subdomains

[zackseuberling]

My site seems to be ignoring my _header file rules, but I don’t understand why. Here’s a sampling of my _headers file:

/*
  Strict-Transport-Security: max-age=31536000; includeSubDomains
  Content-Security-Policy: frame-ancestors 'self'
  X-Frame-Options: SAMEORIGIN
  X-Content-Type-Options: nosniff
  Content-Security-Policy: upgrade-insecure-requests;
  X-XSS-Protection: 1; mode=block
  Referrer-Policy: strict-origin-when-cross-origin

/_next/static/*
  cache-control: max-age=31536000
  cache-control: immutable

After a successful deploy, I run a curl command to see the headers. I am also checking the headers in Postman and in my browser developer tools. This is what I see:

$ curl -I https://development.playco.org
HTTP/2 200
accept-ranges: bytes
age: 0
cache-control: public,max-age=0,must-revalidate
cache-status: "Netlify Edge"; fwd=miss
content-type: text/html; charset=UTF-8
date: Thu, 23 May 2024 00:07:21 GMT
etag: "628d719406f945f182d90093307f1f76-ssl"
server: Netlify
strict-transport-security: max-age=31536000
x-nf-request-id: 01HYHCC4RPW74VSQBCJ18D6EJ0
content-length: 83099

I don’t see any of my top-level /* header rules applied. When I curl assets in the /_next/static/* subdirectory, I do see some header rules applied, like my cache-control header, but a seemingly different strict-transport-security header value.

$ curl -I https://development.playco.org/_next/static/css/dda531369af0463b.css
HTTP/2 200
accept-ranges: bytes
age: 1
cache-control: max-age=31536000,immutable
cache-status: "Netlify Edge"; fwd=miss
content-type: text/css; charset=UTF-8
date: Thu, 23 May 2024 00:13:27 GMT
etag: "864e0027a6c1c813f2e43352faaabc6c-ssl"
server: Netlify
strict-transport-security: max-age=31536000
x-nf-request-id: 01HYHCQA1BY9X03P5VJ0ZAPDS6
content-length: 27634

I have read through as many of the answers I’ve been able to find here including this very helpful one and some other insightful but not particularly helpful ones

The Netlify Site ID for this project is 6e75a6c4-c2d8-4a6e-a529-71558539b623

[dig]

Have you looked at the demos included in the Netlify Next.js Runtime yet @zackseuberling? Specifically the next.config.js in the default demo includes headers which may provide you more clues.

[zackseuberling]

I am using Next.js as an SSG, so I don’t currently use the Next.js runtime. I don’t think the Next.js middleware solution will resolve my issues. Shouldn’t Netlify _headers apply regardless of using a framework middleware?

[zackseuberling]

Interesting discovery, not sure what to do about it. The _headers rules work at the Netlify subdomain: https://development--chimerical-basbousa-4d9dac.netlify.app/

$ curl -I https://development--chimerical-basbousa-4d9dac.netlify.app/
HTTP/2 200
accept-ranges: bytes
age: 2
cache-control: public,max-age=0,must-revalidate
cache-status: "Netlify Edge"; fwd=miss
content-security-policy: frame-ancestors 'self', upgrade-insecure-requests;
content-type: text/html; charset=UTF-8
date: Thu, 23 May 2024 18:25:09 GMT
etag: "5838f1b3a81a9b1bb0df6615fe93c3a4-ssl"
referrer-policy: strict-origin-when-cross-origin
server: Netlify
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-nf-request-id: 01HYKB68RSVD75YN08VSKZQJ4C
x-xss-protection: 1; mode=block
content-length: 83222

but still do not work on the subdomain pointing to that branch https://development.playco.org:

$ curl -I https://development.playco.org/
HTTP/2 200
accept-ranges: bytes
age: 3
cache-control: public,max-age=0,must-revalidate
cache-status: "Netlify Edge"; hit
content-type: text/html; charset=UTF-8
date: Thu, 23 May 2024 18:25:41 GMT
etag: "4c1eddf17518367cd738b47e4a34c293-ssl"
server: Netlify
strict-transport-security: max-age=31536000
x-nf-request-id: 01HYKB78JY98P51EXYRZDC1ARM
content-length: 83099

Is this documented anywhere? How can I verify that my _headers rules are going to work on the primary domain if I can’t test them in a branch with a similar deploy context?

[zackseuberling]

@fool — I see in another thread that there’s a feature flag that a user was opted into for CDN stuff. could that be a similar issue for me?

[dig]

development--chimerical-basbousa-4d9dac.netlify.app and development.playco.org are not serving the same content.

For instance /_next/static/chunks/main-1f3a77ac1a40912e.js exists on the Netlify branch deploy

$ curl -I https://development--chimerical-basbousa-4d9dac.netlify.app/_next/static/chunks/main-1f3a77ac1a40912e.js
HTTP/2 200
# Rest of output

but not on the custom subdomain

$ curl -I https://development.playco.org/_next/static/chunks/main-1f3a77ac1a40912e.js
HTTP/2 404
# Rest of output

The difference in content is evident in the output you posted in the Content-Length

@zackseuberling

content-length: 83222

vs

@zackseuberling

content-length: 83099

You can also see the content is different in the browser by looking at the Network tab of Developer Tools and seeing the files that are requests (e.g. main-1f3a77ac1a40912e.js vs main-77f242bcb709711e.js)

[zackseuberling]

So what’s going on here? Why are they not serving the same code? I have a CNAME pointing to the development branch, and when I run dig they seem like they should be getting served the same content?

$ dig development--chimerical-basbousa-4d9dac.netlify.app.

; <<>> DiG 9.10.6 <<>> development--chimerical-basbousa-4d9dac.netlify.app.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4763
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;development--chimerical-basbousa-4d9dac.netlify.app. IN        A

;; ANSWER SECTION:
development--chimerical-basbousa-4d9dac.netlify.app. 20 IN A 52.9.166.110
development--chimerical-basbousa-4d9dac.netlify.app. 20 IN A 50.18.215.94

;; Query time: 25 msec
;; SERVER: 192.168.0.35#53(192.168.0.35)
;; WHEN: Fri May 24 18:03:29 PDT 2024
;; MSG SIZE  rcvd: 112

$ dig development.playco.org

; <<>> DiG 9.10.6 <<>> development.playco.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10502
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;development.playco.org.                IN      A

;; ANSWER SECTION:
development.playco.org. 300     IN      CNAME   development--chimerical-basbousa-4d9dac.netlify.app.
development--chimerical-basbousa-4d9dac.netlify.app. 20 IN A 50.18.142.31
development--chimerical-basbousa-4d9dac.netlify.app. 20 IN A 13.57.148.141

;; Query time: 90 msec
;; SERVER: 192.168.0.35#53(192.168.0.35)
;; WHEN: Fri May 24 18:04:00 PDT 2024
;; MSG SIZE  rcvd: 148

[dig]

@zackseuberling

So what’s going on here?

development.playco.org is serving playco.org.

Why is it doing this? Because using branch subdomains is a feature of Netlify DNS. If you wish to use branch subdomains without using Netlify DNS read the following support guide

[Support Guide] How to use Netlify’s branch deploy feature without Netlify DNS

Last reviewed by Netlify Support Staff: May 2024 So - you’ve discovered the concept of branch subdomains and want to use them! But you can’t use Netlify’s DNS Hosting? Never fear, there is a way to do this and this post will explain how in detail! Please note, however, that going this route means missing out on single click setup and all the goodness it includes - for example, getting an SSL certificate configured automatically. We also have Automatic Deploy Subdomains, which allows users to …

[hrishikesh]

@zackseuberling

Shouldn’t Netlify _headers apply regardless of using a framework middleware?

No, that’s an incorrect assumption. Quoting from: Custom headers | Netlify Docs

• Custom headers apply only to files Netlify serves from our own backing store. If you are proxying content to your site or dealing with a URL handled by a function or edge function such as a server-side rendered (SSR) page, custom headers won’t be applied to that content. In those cases, the site being proxied to or the function should return any required headers instead. Visit our docs on edge functions to learn how to configure cache-control headers for edge functions.

Regarding the missing headers, I think @dig has advised correctly.

[zackseuberling]

Thanks for all the help! For some reason I had created an alias pointing the development subdomain to the primary URL.

I had correctly set up my external CNAME DNS (through Cloudflare) to point to the Netlify deploy subdomain. Misunderstanding and misconfiguration on my part between what I expected to happen and what Netlify does. Removing the alias fixed everything immediately.

I’m still not totally sure why the domain alias from the Netlify admin slurped up the traffic that should have been going to the netlify.app subdomain, but I don’t really need to know!

[zackseuberling]

@hrishikesh

@zackseuberling

Shouldn’t Netlify _headers apply regardless of using a framework middleware?

No, that’s an incorrect assumption. Quoting from: Custom headers | Netlify Docs

I misspoke — what I meant was shouldn’t the Netlify headers apply to files generated by a framework? Even if you’re also using framework middleware for other headers?

If I use Next.js and deploy to Netlify, I’d expect the the _headers file would work in addition to using the Next.js’s headers middleware convention. I do think there’d be some weirdness doing the same thing in 2 different places. Maybe I’ll deploy some tester website to experiment.

[hrishikesh]

The previous response still stands. _headers only apply to static files framework or not.