Home
Support Forums

Fix for rerendering for ahrefs

Hi,

My netlify site name is remoteambition

We’ve enabled the prerendering service on Netlify. I have an issue where the bot for ahrefs (an SEO service) is not being recognized as a crawler. This causes it to be served the user-facing single-page app when it requests pages from my site (and messes up any reporting). I saw that you previously enabled this for SemrushBot – can you do the same here?

According to their docs here (AhrefsBot. Learn About the Ahrefs' Web Crawler), they’re the second most active crawler after Googlebot.

They use one of 3 user agent strings…

  • Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)
  • Mozilla/5.0 (compatible; AhrefsSiteAudit/6.1; +http://ahrefs.com/robot/)
  • Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; AhrefsSiteAudit/6.1; +http://ahrefs.com/robot/)

Can you add AhrefsBot and AhrefsSiteAudit to your regexp matcher for crawlers?

Blake

Hi, haven’t gotten a reply in 6 days and I’m wondering if someone from support can help. Thanks.

Hi @nfuser,

We’ve begun the process of adding this UA. We’re now waiting for the developers to confirm and release it into production. We’d let you know once that happens. Thank you for your patience.

This change has rolled out across our CDN

2 Likes

That’s great, thank you for rolling this out.