No trailing slash when using proxies

Hello,

We have a large website that we split up into multiple Netlify sites. Our main site is www.dokteronline.com which we use to handle redirects and proxy to all the 9 (language) subsites.

We have Pretty URL optimization activated on all the subsites, and when visiting a page without a trailing slash on this subsite it gets redirected to the URL with the trailing slash. (ex: https://dokteronline-pt.netlify.app/pt/saude-dos-homens)

The problem is that this does not work on the main site when using proxies. (ex: Saúde dos homens | Testosterona | Disfunção erétil | Dokteronline.com)
I have read that this is a known issue so I am curious if there is any update or some kind of workaround to this. Right now it is hurting our SEO.

Greetings,
Jorik

1 Like

This is interesting. Pretty URLs is not supposed to do that:

Ok, I might have misunderstood that part.

But the problem still remains that the 301 redirects are working when visiting the Netlify site directly, but they are not working when visiting the site when using Netlify proxies.

dokteronline-pt.netlify.app >> redirect is working
dokteronline.com/pt >> [proxy redirect to dokteronline-pt.netlify.app] >> redirect is not working

I believe the redirects in the netlify.app subdomain is because the file being served is /saude-dos-homens/index.html. So technically /saude-dos-homens should return a 404 as browsers would look for /saude-dos-homens.html. But Netllify normalises this and serves /saude-dos-homens/index.html.

The behaviour that you see on dokteronline.com/pt is expected. We’re proxying to the destination URL and are able to serve the file from there. When we normally make proxy rewrites, we respect the 301 header and redirect, but in the case I explained in the paragraph before, the 301 is getting ignored as that’s a 301 set automatically by us and not by the user.

To solve SEO issues, I think you should add canonical tags.

I think that I have observed the same bug.

curl -I https://docs-v5--material-ui-x.netlify.app/x/advanced-components is a 301 to https://docs-v5--material-ui-x.netlify.app/x/advanced-components/ where the underlying file served is /x/advanced-components/index.html

However, when we use a proxy: (/x/* https://docs-v5--material-ui-x.netlify.app/x/:splat 200) then the trailing slash auto redirection does not longer work.
Proof: curl -I https://mui.com/x/advanced-components is a 200.

Hi @oliviertassinari :wave: ,

The two sites you referenced appear to have different asset optimization settings. To resolve the issue, you’ll need to:

  1. figure out which site works the way you want, then go to the other site and
  2. uncheck the ‘Disable asset optimization’ setting in Site settings > Build & deploy > Post processing > Asset optimization
  3. mirror the settings from the working site
  4. recheck the ‘Disable asset optimization’ setting

You may be wondering why you need to disable and then re-enable asset optimization. We have an open bug and disabling and re-enabling is the workaround until we can roll out a fix!

Apologies we don’t have any docs on this, but you may want to check out our support guide on trailing slash behavior to get a better understanding.

Hope this helps and do let us know how it goes! :slight_smile:

I have updated the new sites to have the exact same asset optimizations and redeployed. It didn’t solve the problem. @audrey I think that this could be turned into a bug.

@jorik12345 Did you solve the issue on your end?

Hey @oliviertassinari,

This looks related to an issue we’ve already filed for investigation and we’re waiting on its response to debug this further.

Hey @oliviertassinari,

Sorry for the delay. We think we know what is happening and have an explanation. Let us know if it makes sense to you. TL;DR, what you’re seeing is probably the way you want this to work instead of the other way it would work. Here goes the explanation:

When you use Netlify’s Proxy feature, we respect the headers sent by the destination server. For example, consider the following situation:

  1. You setup https://my-proxy.netlify.app/ to proxy to https://www.google.com/
  2. You visit https://my-proxy.netlify.app/ and expect to see https://www.google.com/.
  3. https://www.google.com/ sends a 301 status code and redirects the client to https://www.youtube.com/
  4. Netlify would respect those headers and send a 301 to redirect to https://www.youtube.com/. Basically, the URL in the address bar would now show https://www.youtube.com/ and not https://my-proxy.netlify.app/.

In your case though, we actually send a 301 on the proxy destination. But, we treat internal proxies a bit differently than external proxies. So, we are able to normalise the URL and send a 200 response. Instead, if we send a 301 like we do for the actual site, the proxy would break and you’d be 301 redirected to the destination URL. This is not something you probably want.

We can add this as a feature request to support handling this 301 differently, but I don’t think there would be any immediate actions about this.

1 Like

Thanks for the update. This makes a lot of sense. There might be an opportunity for the proxy to be smart about trailing slash redirections.

I will wait for a fix.