I’m observing intermittent 404 responses on a proxy-ed branch deployment site on Netlify. My issue is very much similar to this issue. I can’t comment since the post is locked.
We have a production site deployed at tigera.netlify.app (hosted via GitHub.). We also have a branch deployment site for a particular version of our product hosted at https://release-calient-v3-2--tigera.netlify.app/v3.2/. We use netlify proxy to proxy requests coming to /v3.2/ of the production site to said branch deployment site. Initially while setting up the site there were some mistakes in deployments which rightfully caused 404 on some paths of the /v3.2/ of proxy (this is because the underlying branch deploy site didn’t deploy properly).
Problem is those 404 responses are cached and intermittently getting displayed (even after the branch deployment succeeded).
I tried clearing the cache manually on deployed the production site.
Did multiple deployments of the branch deploy site hoping to clear the cached responses.
Cleared browser cache and tried loading the URL in incognito mode.
x-nf-request-id: ebd3b849-2404-49e4-82a6-6407e0a9a5fb-5929631 and 6576c810-3d5f-4bd6-80d6-5ee774e37d9e-5774339 for couple of such 404 responses
How do I reliably clear the cache in this case. Sample affected URL here
I wouldn’t proxy to branch subdomains, please use a 301 redirect instead. Those are “special” and proxying is hard for them (can result in situations such as the one you observed). You could alternatively create a separate netlify site with the branch deploy as its production deploy and link to it directly, for best behavior.
I have attempted to clear the proxy cache for you, but if you’d please redeploy BOTH sites - the proxy from and the branch you are proxying to - it will likely clear things up for you in case my clear was not successful.
I’ve set a flag on your account that should make this work reliably going forward, if you take my advice about moving the proxy’d-to branch to its own site with its own custom domain.
Thanks for the reply @fool. I want to get some clarity on what you said about the 301.
Could you elaborate on what side effects this could cause (proxy to branch subdomains) ? We were planning on using this proxy + branch deploy as a long term solution to out multi version product. Any explanations/suggestion here will help us a lot. We might potentially roll back our changes if this might become a problem in the long term.
We didn’t to go with (branch deploy + 301) as the URL becomes not so user friendly and becomes branch name dependent, whereas (200) proxy redirects preserve the pretty URL. But based on what trade offs we have to make for using branch deploy + proxy we might consider going this way / roll back.
Also, just for clarity, the flag you set on our account would make the proxy + branch deploy work consistently Or the 301 rewrites ?
Also, unfortunately deploying both sites doesn’t seem to reset the cache. x-nf-request-id : for the latest 551c8176-2536-4d99-8f57-dc801ad84d11-2280717
We do similar, but proxying to a to a development develop—sitename.netlify.app branch with our own .dev being the proxied dns. We are proxying from one netlify site to 2 or more netlify sites.
@moop-moop, there doesn’t seem to be a question in your post. If you are having issues, would you please send us the x-nf-request-id header for a failed proxy request?
@suresh, for the x-nf-request-id above I show the request occurring at 1:38 pm PDT today.
@fool enabled the feature at 4:46 pm PDT on the 27th. The setting he enabled is that all the sites on the account have their caches reset when any of sites on account is deployed. This ideally allows for redirect rules which proxy to other sites clear their caches when a change occurs on some other site.
However, at the time the request above was made (for x-nf-request-id: 551c8176-2536-4d99-8f57-dc801ad84d11-2280717), the branch being proxied to for /v3.2/* had not been deployed since 4:13 pm PDT on the 27th. The change @fool made therefore didn’t apply (or, at least may not have applied) to the branch subdomain.
In other words, the production branch had been redeployed but the branch subdomain branch had not been.
That being said, I do see that the request for the x-nf-request-id was a 404 when it should not have been.
I do see that branch having been deployed again now (at 5:26 pm PDT). Would you please let us know if you are still seeing any issues with that proxy rule since 5:26 pm (and if so what the x-nf-request-id is)?
Thanks for getting back to me @luke. Our use case has become more confusing now.
We removed the proxy to branch subdomain now and using one monolith site at tigera.netlify.app (which contains a sub folder v3.2 so that we can access the site at tigera.netlify.app/v3.2/). This is what we had previous to (proxy + branch subdomain) and it used to work fine. However even after reverting we are getting the same response as before (the 404 response we got from before with proxy + branch subdomain). This happened on an incognito window. Also do note that this happens only on some URLs intermittently.
x-nf-request-id for the same is: 4b1baaef-7763-46ff-96a2-6e3920340602-12060261 with headers date: Thu, 23 Jul 2020 22:07:13 GMT, etag: 1535491819-ssl-df along with 4b1baaef-7763-46ff-96a2-6e3920340602-12107626 and 91ddb06c-0a7d-44b1-ae42-821302bb7ab9-7185161
I think this is caused by proxy aggressively caching our old response (on some edges at least if not all), is there anyway we can reliably remove that cache ? I tried deploying multiple times… but that doesn’t seem to work much in this case.