Redirect triggering for existing content, erratically

Support
,
#1

Hi,

I have this site hosted on Netlify (https://youthful-wescoff-043c05.netlify.app) which is taking the place of an existing site. As such I wanted to directed any missing content on the new site to the old one. So I added the following redirect:

[[redirects]]
  from = "/*"
  to = "https://old.abolbrasil.org.br/:splat"
  status = 301
  force = false

Which seemed to work great, until I got a report that some images were broken. After looking into it in the reporter’s computer I noticed that the image was actually being redirected as per the rule above. But in my computer this wasn’t happening. After a while investigating I managed to get the same issue on varying URLs, in different computers. The thing is, the files exist and in fact I can access them normally from most computers, browsers, and even CLIs (like curl) I tested.

It seems to me like Netlify is caching a redirect but only on a few edge nodes (totally speculative, it’s just my impression).

What could possibly be happening?

#2

@haggen Welcome to the Netlify community.

I did a quick check of your site and didn’t see any broken or missing images. Typically, if they load on your computer but not on others’ computers, you are seeing the results of local caching. I’ve never seen Netlify cache anything that could cause this to happen.

Can you provide an example of a page with broken images?

#3

Hey @gregraven, thanks for the response!

It’s not just images. It’s whole pages, like, any URL. But it’s very erratic. For me, in my computer (and I tested with several different browsers and even curl in the Terminal) the site mostly works, like 99% of the time.

But very occasionally I get a redirection. I also tested from one of my servers on Digital Ocean and managed to get the issue once. Only once though.

Every time I got the issue though, it was a matter of waiting 10-15 minutes and it would sort itself out, like everything was okay. But then from another computer in my network it almost always get a redirection like the actual file couldn’t be found. It even happened with the homepage (https://youthful-wescoff-043c05.netlify.app).

I feel like I’m not explaining it very well, let me tell you like this:

I would hit an URL, say https://new.abolbrasil.org.br/posts/cabotagem-no-radar-de-operadores-logisticos/ from my computer and it would work just fine. But then on some other computer it would get a redirection to https://old.abolbrasil.org.br/posts/cabotagem-no-radar-de-operadores-logisticos/ (see the subdomain old instead of new). I could refresh it several times and it would get redirected 100% of the time. But if I came back to my PC it would work just fine. Then after 10-15 minutes if I tested the same URL again on the computer that showed the issue it would work just fine.

Intrigued I started browsing every link in the site on my computer (the one that work) and very occasionally I would see the issue and get redirected to the old. subdomain. Then again, if I waited long enough it would go away.

#4

@haggen That page seems to load fine for me, as does the rest of your site, as far as I can tell.

Have you tried using a new private / incognito browser window? This can sometimes get you around local caching issues.

At any rate, as of now, I’m not seeing a problem from here in Southern California.

#5

Hi @gregraven. Thank you for your help.

Yes. I have tried using incognito sessions and different browsers, in different computers, including Opera with VPN. I also tried curl in the Terminal and it got redirected once for existing content. It’s not local cache. The site never had the URL’s being redirected missing, so no browser could ever cache a redirection. I had just setup the redirection rules AFTER having the site deployed for weeks now. As I said, even the root path got redirected once.

There’s no consistency or pattern whatsoever between network, client, computer or geographic region. I tested it all. As I mentioned in my first post it feels like an edge node is poisoned and is misbehaving so most of my hits would work and only a few, occasionally would get the bug. In any case it definitely seems like to be a server thing. Either that or I’m misunderstanding how redirection works in Netlify.

#6

@haggen If you’re certain it’s not a caching issue, then download the deploy from Netlify and verify that each of your assets is where you think it is, and that each of your links to those assets is correct.

1 Like
#7

@gregraven I’m quite sure they are because at the same time I get the issue in one computer or in one browser at the same computer, I can access the same URL successfully, but from a different browser or computer.

I just tested it again and I’m still having the issue but only on a MacBook and when using Safari. If I access any of the URL below in Chrome in the same MacBook the redirection doesn’t happen.

But as I said earlier I did get the issue on different URLs, with different clients in different computers, just erratically, one time it would redirect, 10 minutes after it would access it just fine.

#8

If you manage to replicate this, if you can grab the x-nf-request-id or a HAR file, we’ll be able to home in!

#9

Okay now I’m not getting the issue anymore, not even on the MacBook.

#10

@Pie Gotcha! X-NF-Request-ID: b8b3478b-4df5-4f51-96ba-2b2da93e91e6-38969652.

The URL is https://new.abolbrasil.org.br/cms/. I downloaded the build and checked; the file ./cms/index.html is there, but I’m getting redirected even from the Terminal.

➜ http -h https://new.abolbrasil.org.br/cms
HTTP/1.1 302 Found
Age: 0
Connection: keep-alive
Content-Length: 49
Content-Type: text/plain
Date: Wed, 02 Sep 2020 21:34:30 GMT
Location: https://old.abolbrasil.org.br/cms
Server: Netlify
X-NF-Request-ID: b8b3478b-4df5-4f51-96ba-2b2da93e91e6-38969652

Update 1: Oh now the whole site is being redirected, like any URL I hit I get redirected.

➜ http -h https://new.abolbrasil.org.br/
HTTP/1.1 302 Found
Age: 0
Connection: keep-alive
Content-Length: 46
Content-Type: text/plain
Date: Wed, 02 Sep 2020 21:43:28 GMT
Location: https://old.abolbrasil.org.br/
Server: Netlify
X-NF-Request-ID: b8b3478b-4df5-4f51-96ba-2b2da93e91e6-39360979

Update 2: It seems to be only from my network (tested from different computers). From my server I’m getting though just fine.

#11

Hey @haggen,

You’ll be glad to know that I was able to spot the problem with the information provided :partying_face:!

One of our dozens of nodes, in São Paulo, had a cache discrepancy. I was able to drop the cache for your site and, from my tests, all appears well. Can you monitor this and feed back if the issue reoccurs, please?

1 Like
#13

:star_struck::balloon::tada::confetti_ball:

Thank you so much, @gregraven and @Pie, for your support! Tonight I’ll drink to your health! :beers:

2 Likes
#14

@Pie Bad news! :disappointed: It’s happening again, here’s the ID: x-nf-request-id: b8b3478b-4df5-4f51-96ba-2b2da93e91e6-66857625.

If you need anything else let me know! And thanks again!

#15

Ah! Looks like you’re proxying to us from Cloudflare. This is almost certainly the problem and here’s the reason why (and remediation)!

#16

@Pie I’m using CloudFlare but for DNS only. I have never enabled CloudFlare edge services for this domain.

image
image959×54 2.2 KB

Even being DNS Only could CloudFlare still be intercepting the requests?

#17

@gregraven @Pie Hi guys! I don’t want to sound ungrateful, but could you guys take another look at it? I was supposed to have the site released this week and my client is getting tired of hearing “we’re having technical difficulties with one of our providers”. :sweat_smile:

I triple-checked and the request is not going through CloudFlare. I have only ever used them as DNS, so the requests have always been straight to Netlify. You can confirm by resolving the domains youthful-wescoff-043c05.netlify.app and new.abolbrasil.or.br and checking they translate to the same IP.

The issue stays the same, the redirection rule is being triggered for URLs that match existing content in the build.

One of my computers is getting redirection on ALL URLs since yesterday. Root, static, files, HTML, whatever I hit I get redirected, as if the build was empty. But this only happens on one of my computers, even via curl. On my other computers it seems to be working.

Once again, thank you for all your support!

#18

@haggen Sorry you’re still having an issue.

I went through your site a few days ago, but everything seemed to work. Of course, I don’t know what I’m looking for. What’s precisely is not working for you and precisely what have you tried that is not giving you the results you expect?

Also, I find it confusing that most of your computers deliver your site without problems, and only one doesn’t. That seems to imply that the code is correct, and there is something different about that computer.

What am I missing?

#19

@gregraven Hi, thanks for your response.

The exact problem is that I have setup a redirection rule that, according to docs, should only trigger if the request URL doesn’t match an existing content in the build. Before that rule the site was working as expected. But once I added the rule I started to get redirected for existing content, only intermittently. So, sometimes I got redirected, sometimes I didn’t and that indicated to me that the files were there, it was the server that was misbehaving.

I followed through some debug steps.

  • Made sure Netlify was the one handling my requests
  • Checked the actual build content for the files that were being treated as missing, even though I had evidence that they weren’t the problem.
  • Reproduced the issue from multiple browser, computer and networks. So I tested and stumbled upon the issue using Chrome, Safari, Firefox, on Window, macOS, and even using curl from one of my servers (DigitalOcean’s USA3 region).

But due to the erratic behavior of the issue it’s difficult to establish a pattern or even collect evidence.

In any case, @Pie have actually identified an issue with one of the cache nodes, which makes a lot of sense and was one my suspicions from the start. I have had trouble with cache in the past (not on Netlify though), so it’s not my first rodeo.

The issue seemed to have been sorted out until earlier today when it happened again. It’s weird because some computers seem to always or almost always get the issue, but some other rarely do. But then again, it makes sense due to nature of distributed networks.

Above you can find a few request IDs I have taken note. Those were requests that triggered the redirection rule even though the matching content was present in the build files. I checked.

I’m out of ideas from my side, and as Pie have stated, there was an issue with Netlify servers.

If you need anything else let me know.

Thanks again for everything.

#20

@haggen OK, that’s a start, but I was hoping for some specific information in addition to the generalities.

What URL(s) are you looking at?

What’s happening currently when you visit those URLs?

What do you want to happen when someone visits those URLs?

What have you tried so far, as in, what do your redirects look like?

Finally, your last post seems to contradict in part your earlier post that stated:

One of my computers is getting redirection on ALL URLs since yesterday. Root, static, files, HTML, whatever I hit I get redirected, as if the build was empty. But this only happens on one of my computers, even via curl. On my other computers it seems to be working.

So is it just one computer on one network, or various computers on various networks?

#21
  • What URL(s) are you looking at?
    Anything on https://new.abolbrasil.org.br. Literally any URL on this domain is susceptible to the issue. I say susceptible because as I said several times the issue is erratic, meaning it happens without a pattern. Sometimes it’s only one or two URLs that redirect. Other times it’s all of them. Other times it’s none of them and everything’s working as expected. Sometimes the issue lasts 15 minutes and no matter how many requests I make to an URL it gets redirected. Other times it lasts the entire day. Some of the computers I tested experienced it more than others, but once again without clear pattern or cause. Bottom line, I have experienced the issue at least once across all computers and networks I tested from.
  • What’s happening currently when you visit those URLs?
    I get redirected (response status 302) as if the content was missing from the build files, according to the redirecting rule I setup on my netlify.toml, e.g. I hit https://new.abolbrasil.org.br/ and get redirected to https://old.abolbrasil.org.br/.
  • What do you want to happen when someone visits those URLs?
    I wanted to get a response with status 200 OK and see the homepage of my new website. Screenshot for reference.
  • What have you tried so far, as in, what do your redirects look like?
    I have requested several different URLs from the domain new.abolbrasil.org.br that match existing files in the build content. From different computers, using different clients (browser and CLI) and have detected the issue at least once from all of them, no matter the combination. Here’s an example using httpie (a command-line HTTP client).
    ➜ http -h https://new.abolbrasil.org.br/cms
HTTP/1.1 302 Found
Age: 0
Connection: keep-alive
Content-Length: 49
Content-Type: text/plain
Date: Wed, 02 Sep 2020 21:34:30 GMT
Location: https://old.abolbrasil.org.br/cms
Server: Netlify
X-NF-Request-ID: b8b3478b-4df5-4f51-96ba-2b2da93e91e6-38969652
    If I download the build files and go look for /cms/index.html it’s there. I can confirm it by accessing the URL from other computers or browsers or simply waiting some time until one of them actually returns the expected content.
  • So is it just one computer on one network, or various computers on various networks
    It’s all of the above. Since the issue is intermittent I don’t always experience it. But if I test it enough, meaning, if I leave a script requesting URLs from my site I eventually get the issue and receive a redirection instead of the actual expected file. But I also noticed that some computers experienced it more than others. But I think it was just a coincidence since I have no evidence connecting a characteristic of the client with the issue.

I hope it’s a satisfying summary of what’s happening and you can move forward with your investigation. I understand you’re just following procedure but I’ll say once again that @Pie have stated there was in fact an issue with the cache servers, which it makes sense from a technical perspective. Cache servers are load balanced following some algorithm that takes region into account, for instance, which would explain why my server on US was the least affected across all my tests. It also explains the erratic nature of the error. Sometimes I would hit the defective server, other times I would hit a healthy server.

Thank you and I’m looking forward to have this issue resolved.