502 and 504 errors in ahrefs - Netlify Edge fwd miss - How to diagnose and fix?

Site name - cool-kleicha-4d293e
Site ID - da43895b-abbe-4395-ab00-a32697da7ab0

Hey all,
Since the week of 24th Jan I’ve been seeing 502 and 504 errors in my weekly ahrefs report. This is having a negative effect on the site’s health score, which hurts SEO of the site.

The number of pages with 5XX errors fluctuates (between 4 and 12 pages) but it’s happening every week, and it’s not the same pages each time.
The site has been live without these issues for several months, so I’m not sure what to look into to find the cause.

I looked at the http headers and I’m seeing Cache-status “Netlify Edge”; fwd=miss

I asked the Ask Netlify AI and it said X-nf-request-ids might be useful. Here’s 2 from this week’s ahrefs report:

Here’s a 504 from today:

Status HTTP/2 504
Age 28
Cache-control private,max-age=0
Cache-status “Netlify Edge”; fwd=miss
Content-type text/plain; charset=utf-8
Date Fri, 21 Feb 2025 15:18:42 GMT
Server Netlify
Strict-transport-security max-age=31536000
X-nf-request-id 01JMMHCYSYA2QVV9MTH5NJGVT0
Content-length 0

Here’s a 502 from today:

Status HTTP/2 502
Age 5
Cache-status “Netlify Edge”; fwd=miss
Content-type text/html
Date Fri, 21 Feb 2025 15:17:33 GMT
Server Netlify
Strict-transport-security max-age=31536000
X-nf-request-id 01JMMHBHVXYS0JHNC2TGQ429Q0
Content-length 524

I have plenty of bandwidth and the site is static, so no fancy stuff going on.
Everything is deployed and hosted on Netlify, so I’m not sure what to do to diagnose further for a fix.

I feel like this must be something obvious I’m missing or misunderstanding. Any help would be appreciated.

Try disabling prerendering: Build & deploy | Site configuration | cool-kleicha-4d293e | Netlify

1 Like

Yep, that seems to have solved it. I’ve disabled prerendering, and run another site audit in ahrefs and now I’m getting zero 5XX errors.
Thanks @hrishikesh

Why does this happen? I have the same issue now on all my sites. Is it just Ahrefs or does Google bots also get this error?

I was curious so I checked my two other sites, and I was getting the same 5XX error problem in ahrefs.

All three sites are built as static sites with Astro, and all 3 had prerendering set to On in Netlify, which for me is showing as beta. So maybe it’s some conflict between Astro static sites and Netlify prerendering?

@hrishikesh Are you able to shed any light here so we know what to look out for.

Astro generates a static site, why do you need prerendering on it?

Exactly. I think that’s maybe what is causing the error.
In some cases Astro builds static sites, which wouldn’t need prerendering. Prerendering was set to on by default on my sites. I switched prerendering off, and the 5XX errors stopped.

But, not all Astro sites are static as Astro also has SSR mode. So maybe prerendering is still useful to those sites, I don’t know.

In an ideal world it feels like Netlify should detect that prerendering isn’t useful for the project, and therefore not default to on.

Those are rendered on the server, and thus don’t need prerendering either. Only CSR sites need prerendering, the ones developed with Create React App, or Vite natively.

Netlify by default doesn’t prerender your site. If you turn it on, I don’t see a reason why Netlify should still try to be smart about whether or not your site needs it. We trust that the developer of the site would have a good reason to turn it on.

Hmm interesting. All 3 of my Astro sites had prerendering set to on, and I’m fairly certain I didn’t switch that on myself. I didn’t even know that was an option in Netlify. Could this be something getting switched on automatically as part of the initial setup?

I use Nextjs in my most of my projects, and some of the sites DO benefit from prerendering, but it causes the 5XX problem. I still don’t understand why? And I have not switched on the prerendering, it has automatically been enabled on all my sites, also sites I haven touched in a couple years

How so? Won’t Next.js render everything on the server? If not then there’s somethinng wrong with your Next.js setup.

No. We have no reason to enable a beta feature by default especially since it can cause issues.