Google Search Console says robots.txt is inaccessible although robots.txt returns HTTP 200

Support
1

Hello,

I’m hosting my website on Netlify.

Site:

Google Search Console refuses to index my homepage because it reports:

“robots.txt inaccessible”

However, everything appears to be working correctly.

What I have already verified:

  • HTTPS works correctly.
  • Let’s Encrypt certificate is valid.
  • Domain DNS is configured correctly.
  • robots.txt is publicly accessible.
  • sitemap.xml is publicly accessible.
  • Search Console domain ownership has been verified.
  • robots.txt returns HTTP 200.
  • Homepage returns HTTP 200.
  • Googlebot User-Agent also receives HTTP 200.

Tests:

curl -I https://rodyengenharia.com.br/robots.txt

HTTP/1.1 200 OK

curl -I -A “Googlebot” https://rodyengenharia.com.br/robots.txt

HTTP/1.1 200 OK

curl -I -A “Googlebot” https://rodyengenharia.com.br/

HTTP/1.1 200 OK

robots.txt:

User-agent: *
Allow: /

Sitemap: https://rodyengenharia.com.br/sitemap.xml

The Rich Results Test and Google Search Console still report that robots.txt cannot be accessed.

Could someone please verify from the Netlify side whether Googlebot requests are reaching Netlify Edge correctly or if there is any issue affecting Google’s crawler?

Thank you.