Home
Support Forums

Only getting 404 through proxy

site: https://dreamy-torvalds-a57a18.netlify.app/

The site has two endpoints: /index and /console

I’m able to access the site directly, but when I try to go through a reverse proxy (haproxy) I am only getting 404. I understand that proxies aren’t officially supported, and I’m not asking for help configuring the proxy. I simply want to understand what is wrong with the requests being made by the proxy.

Here are two request ids: 01FFK0NAJ0STTP18C0T13X7VJ8, 01FFK179Y858N433SFW5HD34YJ

Thanks

Hi, @edavidson. I looked for those request ids and I don’t see them in our system. It is possible there is a service worker for that site on your local browser? I ask because this is the most common reason people see x-nf-request-ids that we have no record of.

What happens is that case is that the service worker is returning an old HTTP response which includes an x-nf-request-id which is from over 30 days ago and therefore isn’t in the stored logs (which only go back 30 days).

We have more information about service workers here:

Hi Luke, thanks for the response. There were no service workers or caching involved, and those requests were within 30 days.

The issue actually turned out to be that the proxy wasn’t setting the ‘host’ header. Once I added that header it is working as expected.

1 Like

Thanks so much for coming back and letting us know! Glad everything is working as expected :netliconfetti:

To hopefully clarify the issue for anyone else that may come across this, the only way any web server responding to HTTP requests can know the domain name being requested is the host header. If that header is missing, then you are making a request to an IP address and passing a path but not telling the web server what the domain name is.

Note, in some cases, knowing the domain name and sending that header is not required. Some stand alone web servers do work this way and you can make HTTP requests without that header. I’ve certainly run internal-only wikis that were hosted at http://10.0.0.1/wiki and not at a domain name.

However, Netlify always requires that header. Netlify’s CDN has many websites hosted on it and we rely on that header to know which site to respond with. If you leave the header off entirely, we send a 404 as without it we don’t know what page you are looking for.

A final note, I did testing of our CDN without that header and all my x-nf-request-id header values (and the 404 status response itself) do get logged. It is still quite a mystery why I cannot find 01FFK0NAJ0STTP18C0T13X7VJ8 and 01FFK179Y858N433SFW5HD34YJ in our logs. I believe you did receive those values. I just cannot explain why I cannot find them in the logging.

2 Likes

Thanks for clarifying this! It makes much more sense now.

On the mystery request IDs: the proxy was probably keeping the host header from the original request, so it would probably be localhost, rather than not having any host header. idk if that makes a difference.