Wrong 301 Redirect for percentage encoded URLs

I have this url in my website that get a 301 response, but I think it is incorrect:

https://mekke.guupa.com/book/%e3%81%9f%e3%81%b9%e3%82%82%e3%81%ae%e3%81%ae%e3%81%9f%e3%81%b3/

It is redirecting to

https://mekke.guupa.com/book/%25e3%2581%259f%25e3%2581%25b9%25e3%2582%2582%25e3%2581%25ae%25e3%2581%25ae%25e3%2581%259f%25e3%2581%25b3/

that is a double-encoded URL.

I modified my website to do a double-decode, so the page is still working, but the unnecessary 301 redirect is creating SEO issues.

This is the original URL: https://mekke.guupa.com/book/たべもののたび

I tested using

curl -I -i https://mekke.guupa.com/book/%e3%81%9f%e3%81%b9%e3%82%82%e3%81%ae%e3%81%ae%e3%81%9f%e3%81%b3/

curl -I -i https://mekke.guupa.com/book/%25e3%2581%259f%25e3%2581%25b9%25e3%2582%2582%25e3%2581%25ae%25e3%2581%25ae%25e3%2581%259f%25e3%2581%25b3/

With the first curl command there is a 301 response. It should instead send a 200.

The second curl return a 200 response, but the url is double-encoded. I don’t understand this logic.

Any help would be appreciated

@lucamug Welcome to the Netlify community.

This URL seems to load OK. Are you getting your 301 response only with curl, or how?

When I try to test this URL with curl --head --location I get an error message:

curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)

However, the page does load so this may be an issue with curl.

Thank you for the reply. Let me paste the complete response to make it clearer:

$ curl -I -i https://mekke.guupa.com/book/%e3%81%9f%e3%81%b9%e3%82%82%e3%81%ae%e3%81%ae%e3%81%9f%e3%81%b3/

HTTP/2 301
cache-control: public, max-age=0, must-revalidate
content-length: 0
content-type: text/html; charset=UTF-8
date: Sun, 16 May 2021 12:27:47 GMT
etag: "2b4a62efe10fbe29bcd904fb43240453-ssl"
strict-transport-security: max-age=31536000
server: Netlify
location: /book/%25e3%2581%259f%25e3%2581%25b9%25e3%2582%2582%25e3%2581%25ae%25e3%2581%25ae%25e3%2581%259f%25e3%2581%25b3/
age: 0
x-nf-request-id: 1050c765-b59f-4181-a660-8a20aeac5378

$ curl -I -i https://mekke.guupa.com/book/%25e3%2581%259f%25e3%2581%25b9%25e3%2582%2582%25e3%2581%25ae%25e3%2581%25ae%25e3%2581%259f%25e3%2581%25b3/

HTTP/2 200
cache-control: public, max-age=0, must-revalidate
content-length: 0
content-type: text/html; charset=UTF-8
date: Sun, 16 May 2021 12:27:58 GMT
etag: "2b4a62efe10fbe29bcd904fb43240453-ssl"
strict-transport-security: max-age=31536000
age: 0
x-nf-request-id: 9ec1ca33-a2d0-49d7-987b-b6691b3097a4
server: Netlify

The issue being that the first reponse is 301 and the second is 200. Both should be 200.

The error that you get I think is if you use the original Japanese url. That is not a valid url, it need to be percentage-encoded.

OK, but my question was that if the page is loading fine, what difference does it make if curl can’t figure it out?

Issues are:

  • SEO: Google doesn’t index 301 redirect
  • The url is working fine because I created an hack in the code that does a doube-decode (the target of the 301 redirect is a wrong URL, so by default it would get “page not found” error).
  • If confirmed, it is a wrong redirect that can cause issues on other websites too

The target of the 301 redirect (location: /book/%25e3%2581%259f%25e3%2581%25b9%25e3%2582%2582%25e3%2581%25ae%25e3%2581%25ae%25e3%2581%259f%25e3%2581%25b3/) is a wrong url.

This issue may effect all websites that have non-ASCII characters in the url and use percentage-encoding.

As a test, I published the site using Surge, and the response is a correct 200. So the issue seems not related to the URL itself, but to some server configuration.

curl -I -i https://mekke.surge.sh/book/%e3%81%9f%e3%81%b9%e3%82%82%e3%81%ae%e3%81%ae%e3%81%9f%e3%81%b3/

HTTP/1.1 200 OK
Server: Surge
Surge-Cache: HIT
Surge-Stamp: 10084::1621168891141-48c23cf4e08f73c5a89386f0f863fca1
Age: 196
Date: Sun, 16 May 2021 12:50:23 GMT
Cache-Control: public, max-age=0, must-revalidate
ETag: "48c23cf4e08f73c5a89386f0f863fca1"
Content-Type: text/html; charset=UTF-8
Accept-Ranges: bytes
Response-Time: 1819ms
Content-Length: 117791
Vary: Accept-Encoding
Connection: close

Hey there, @lucamug :wave:

It looks like this thread has been a bit quiet for the past few days-- thanks for your patience. Are you still encountering this issue? If so, please let us know what additional debugging steps you have taken since you last wrote in.