Prevent Google from indexing netlify URL

Support
,
#1

After connecting a custom domain, how do we prevent Google from indexing the netlify URL (example.netlify.app)?
I’m using Netlify DNS.

Can someone confirm these are the steps?

Step 1. Save a plain text file called _redirects in my root directory. (Note: my project is just an index.html file in my root directory. Not using a build command or static site generator).
Step 2. Inside of the _redirects file write:
https://example.netlify.app https://example.com 301

Is that it? Do I need to write the http and www versions of these lines too?

Is the netlify.toml file required too?

Thanks

#2

Hi @flynet,

You could try using a custom headers rule in a _headers file, like this:

https://example.netlify.app/*
   X-Robots-Tag: noindex

This will tell google not to index that url at all. Let me know if that works for you.

#3

I created a _redirects file based on the solution described here. The file says:

https://example.netlify.app https://example.com 301!
http://example.netlify.app  https://example.com 301!

Do I also need to add the custom headers rule? According to this Google article: "If you need to change the URL of a page as it is shown in search engine results, we recommend that you use a server-side 301 redirect. "

Please clarify if the current setup is enough or if the _headers file is required too.

Thank you!

#4

I followed the solution here.

Do I also need the _headers file? This Google article suggests that 301 is enough to change the URL in search results.

Please advise. Thank you.

#5

Hi, @flynet, the 301! rules will force all traffic to the custom domain. This alone is enough to prevent Google from indexing those pages.

However, if you have a site without a custom domain, the _headers rules are another way to prevent indexing. This can be helpful when you have a demo/testing site without a custom domain and you don’t want it indexed.

There is a third method as well. The “robots meta tag”:

https://developers.google.com/search/reference/robots_meta_tag

To summarize, the 301! rules can only be used with custom domains. The X-Robots-Tag HTTP header and robots meta tag can be used on any sites, custom domain or not.