Intermittent ERR_TOO_MANY_REDIRECTS when url contains unicode

[Blogger238]

Site name: https://rpdebug.netlify.app

No DNS issues.
NO build problems.

Everything was working fine until yesterday, when all of a sudden, I saw this site cycling back and forth between these symptoms:

1. Works fine on all Windows/Firefox, Windows/Edge, and Android/Chrome
2. Works fine on desktop, but ERR_TOO_MANY_REDIRECTS on Android/Chrome
3. Works fine on Windows/Firefox, but ERR_TOO_MANY_REDIRECTS on Windows/Edge and Android/Chrome
4. ERR_TOO_MANY_REDIRECTS on all browsers and platforms.

There are no redirects in the site content. I also deleted the site and recreated it newly, and the same problem exists.

Here are a few examples of requests that resulted in ERR_TOO_MANY_REDIRECTS:

< x-nf-request-id: 01FJ3Y79P2D53CZFSMC7AJJ3PX
< x-nf-request-id: 01FJ3Z4P9VH2R3MKXXRAQ2QZDJ
< x-nf-request-id: 01FJ3ZN5FFJ717GV9WN5C52ZC2

The top level site always functions fine, but any page within it does not. The above request was made to:

https://rpdebug.netlify.app/budding-rasik%C4%81/teaching-your-child-to-become-a-carn%C4%81tic-music-rasik%C4%81/

Help would be most appreciated, thank you!

Edit: changed site domain to ‘rpdebug’.

[Blogger238]

Aha: the problem seems to occur only on pages which include unicode characters in the url. At least, I haven’t been able to reproduce the problem on other pages so far.

Is it possible that some servers on the CDN are not handling unicode chars properly?

[hrishikesh]

Hi @Blogger238,

Interestingly, I’m not getting any 301s for that page. Are you still seeing this?

I’ve also purged your website’s cache from our end, in case that changes something?

[Blogger238]

Thank you, @hrishikesh. Yes, I’m seeing it even right now, on Android/Chrome.

The problem is intermittent. The error cycles on and off every few minutes or hours on the desktop.

A test page I made on the same site without unicode in the url doesn’t have this problem, and has remained constantly on:

https://rpdebug.netlify.app/_notes/xyztest/index.html

An example of a url with unicode that errors intermittently:

https://rpdebug.netlify.app/budding-rasik%C4%81/teaching-your-child-to-become-a-carn%C4%81tic-music-rasik%C4%81/

My other sites on netlify without unicode in the urls work fine. I also created a copy of this problem site replacing the Unicode characters with non Unicode and that one has been fine for the last 15 hours as well.

[coelmay]

I have managed to replicate (I believe) what @Blogger238 is seeing in that I was able to load the page

https://rpdebug.netlify.app/budding-rasik%C4%81/teaching-your-child-to-become-a-carn%C4%81tic-music-rasik%C4%81/

locally, but now cannot. Unfortunately I didn’t get the x-nf-request-id for these. I did however use a VPN to connect to various locations to test.

UK failed first time. Last X-NF-REQUEST-ID: 01FJ5EN3BTW1T46DAASSD25YA1 before Firefox failed.

---

NY Worked first time: X-NF-REQUEST-ID: 01FJ5ET56NCVVPPZE9DRQ7WH6E

NY Failed on subsequent visit (different VPN IP though.) Last X-NF-REQUEST-ID: 01FJ5F576RK5C3VY8MM8E069FM before Firefox failed.

---

Hopefully this add something useful to the investigations @hrishikesh.

[Blogger238]

That’s exactly the symptom I’m seeing. Thank you, @coelmay!

@hrishikesh, are you able to reproduce this, or investigate the x-nf-request-ids? Thank you.

[hrishikesh]

That’s interesting, thanks @coelmay.

I’ve still not seen this happen myself, however, I’m noticing a pattern - all the request IDs shared by both of you were ‘cached’ in the CDNs.

So I checked further to see all the 301s for that URL in the past 7 days and 1530/1550 were cached. So this might have to do something with caching.

Could you deploy this as a separate website and see if the problem persists?

On a side note, I’m now getting 404 for that URL.

[einar]

I experienced something similar today and was just about to write a post when I saw this.

I was testing my app on my mothers old laptop. When I opened Firefox DevTools I saw 21 (!) redirects (301) after each other, ending up with a warning “too many redirects”. Refresh did not help. The URL includes some swedish characters å, ä, ö. https://cv3-dev.netlify.app/mörbylånga/

Host: cv3-dev.netlify.app
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:52.0) Gecko/20100101 Firefox/52.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: sv-SE,sv;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate, br
Referer: https://cv3-dev.netlify.app/
Connection: keep-alive
Upgrade-Insecure-Requests: 1

Response headers

Age: 342
Cache-Control: public, max-age=0, must-revalidate
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Date: Sun, 17 Oct 2021 07:52:28 GMT
Etag: "08ca3ab8743a9b7a4ce435b5b02b7a27-ssl"
Location: /m%C3%B6rbyl%C3%A5nga/
Server: Netlify
X-Firefox-Spdy: h2
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-nf-request-id: 01FJ6M45AVTM42A4506P8KW46P

The following redirects had the same response headers except the x-nf-request-id
01FJ6M45EHMTZ42YEAKPA30NEB
01FJ6M45H906HD8GQRF0PFSDNM
01FJ6M45MN39TT96MSN9JSS7B9
01FJ6M45TN0G0Z44D3XCBXH7BM
…

20 retries before giving up (21 total)
Age also incremented somewhat (not every retry, ended up on 345)

Suddenly after a new refresh it started working again:
But another IP 206.189.50.215:443 instead of previous 3.67.153.12:443

[Blogger238]

@hrishikesh, a local deployment does not reproduce the issue after extensive testing.

A 98% cache hit rate sounds normal to me, and the fact that the problem occurs on both uncached and cached hits indicates to me that the issue might be elsewhere. My two cents.

To reproduce these, have you tried to access these from an Android?

@einar good to know! Curious, how long have you had these urls deployed for?

@hrishikesh wondering if this is now considered to be a bug and how this issue will be prioritized and worked on? Guidance on this question would be helpful. Thank you for looking into it.

[einar]

“How long have you had these URLs deployed for?”

First deploy Oct 14. Experienced these problems on Oct 17 (maybe earlier as well, I think I saw this on another computer as well briefly).

[hrishikesh]

Hi @Blogger238 and @einar,

Thank you for sharing! We’re filing this for our developers to take a look at. We can see the problem exists so they can investigate it on their end.

We’d appreciate your patience till this gets solved.

[hrishikesh]

Hey all,

The team has released a fix and it should be working correctly at the moment. Could you check and report back?

[Blogger238]

Thank you @hrishikesh and team for the fix!

I’ve checked twice so far, and it’s worked fine. Since the problem is intermittent, I’ll check a few more times later today and tomorrow and get back.

[Blogger238]

Still working fine as of now!

[Melvin]

Hi @Blogger238 @einar,

Thanks for your patience!

I wanted to update you that the issue has been resolved. Please let us know if you do run into the issue again. Thanks!

[Blogger238]

@Melvin, that is great. Thank you very much for the quick resolution!

If at all you could share some technical details about what the problem was, that would be great for all the technical folks on this thread.

Regardless, thanks again!

[fool]

Sure, happy to speak to that to the extent of our knowledge.

Seems like this was a bug in our redirect handling code around unicode-containing rules and potentially even redirects for URL’s that simply had unicode in them that matched a redirect rule.

You experienced it partially since our CDN was running some proposed updates on some nodes which failed in the way you observed, and which we applied a fix for while we were working on another issue. We think the scope of the problem was (again, only on a few CDN nodes) from about 11 through 18 oct or thereabouts.

Hope that helps!

[Blogger238]

@fool, that’s very informative and helpful. Thanks again for the awesome support and engagement on this thread!