Netlify proxying redirect causing duplicated headers

Support
1

Hi! I recently started a project that needs to send POST requests to a legacy API on another domain (I don’t manage the API and can’t modify it). For this reason, Netlify’s rewrite rules (i.e. a redirect rule with status code 200) was the perfect feature to integrate it since I need to avoid CORS. The API needs to receive a Content-Type: text/xml; charset=utf-8 header. Since I’m configuring fetch in no-cors mode, I cannot set this header directly (it’s restricted, see Using the Fetch API - Web APIs | MDN for the allowed headers). Thus, I created the following redirect rule in my netlify.toml file:

[[redirects]]
  from = "[source-url]"
  to = "[legacy-api]"
  status = 200
  force = true
  [redirects.headers]
    Content-Type = "text/xml; charset=utf-8"
    Accept-Language = "unimportant"
    SOAPAction = "unimportant"

The JS code I’m running to send the request is:

  const config: RequestInit = {
    method: 'POST',
    mode: 'no-cors',
    body: '[unimportant XML string]'
  };

  fetch('[source-url]', config);

Now, the fetch client will add a Content-Type: application/x-www-form-urlencoded header since I specified a string as the body parameter.

The API sends a 400 Bad Request response, and rightly so. I changed the [legacy-api] URL to a local URL so I could dump the raw request. Here’s what I got (some headers were hidden for privacy):

--> POST / HTTP/1.1
--> Host: XXX.localhost.run
--> X_Forwarded_For: XXX
--> X_Forwarded_Proto: https
--> X_Forwarded_Host: XXX
--> User-Agent: curl/X.X.X
--> Content-Length: 457
--> Accept: */*
--> Accept-Language: es-es
--> Client-Ip: XXX
--> Content-Type: application/x-www-form-urlencoded
--> Content-Type: text/xml; charset=utf-8
--> Forwarded: for=XXX;proto=https
[etc.]
--> Accept-Encoding: gzip
-->
--> [unimportant XML string]
-->

As you can see, the Netlify server sent a duplicated header. This is undefined behavior per RFC 7230 (Section 3.2.2):

A sender MUST NOT generate multiple header fields with the same field name in a message unless either the entire field value for that header field is defined as a comma-separated list [i.e., #(values)] or the header field is a well-known exception (as noted below).

I would expect the proxy to override the Content-Type header sent by the client (which I cannot change, since fetch sets it by default, and is a restricted header in no-cors mode). The other valid solution (but, imo less useful) is to send the client’s header. However, they cannot be sent at the same time.

My site name is https://crtm-card-status.netlify.app/; the real API route is /status, and I’m using /netdebug with the same headers for testing.

Thanks in advance!

3

Hey there, @hugmanrique :wave:

Thanks so much for reaching out and writing such a thorough post about what you are encountering and what you have tried.

Before I loop in a Support Engineer, I want to make sure you have read the following redirect Support Guide, as well as two other posts from Forums members on duplicated headers and the solution they found. Let us know if these resources help!

4

Hey Hillary!
I’ve read all the troubleshooting steps on the first guide and, unfortunately, none seem to apply here. As for the other two post: the first one deals with Netlify functions, which I am not using here; and the second one looks like it has a similar cause as this problem. It roughly states that adding a header already sent by your servers to a _headers rule that matches can cause duplicate headers. Here the issue is caused by adding a matching rule to the _redirects file (or equivalently, a redirect on the netlify.toml file). Unrelated, but I wonder if both problems are caused by the same code path/data structure. However, the issue given there (uncapitalizing the header name) doesn’t work here.

Thanks for your prompt reply :grinning:

1 Like
5

Hi @hugmanrique,

I believe this might be a limitation of the adding customer headers to proxied content feature. I believe there are no checks in place to see if the proxy destination is already adding some headers and overwriting those - our feature is merely intended to add additional headers that might be required.

6

I understand, but I believe sending invalid HTTP requests from your servers isn’t ideal when the solution (I’m assuming, I can be wrong) isn’t hard. Otherwise, any client that knows the “injected” header names can break proxied requests as they please.

7

That is a valid point. While there’s no immediate solution to this, I can file this as a feature request. However, I don’t know if/when this will be implemented.

1 Like