However, after 2 weeks the site has been published, the sites still not appearing on search engine (both Google and Bing). How is this 200! redirect affect SEO and search engine ability to crawl the webpages?
Other information:
The https://children.wpengine.com/live/ is hosted on WordPress and we have confirmed it should works fine. On Google Search Console, I am not able to request indexing for https://www.parent.com/children/live/ and their bot is not able to crawl the page and received 500 status error instead.
Indexing request reject on Google Search Console because the site returned 500 status error
Yes, I purposely use 200 (rewrite) instead of 301 (redirect). The goal is to have the URL in the visitor’s address bar remains the same, while Netlify’s servers fetch the new location behind the scenes.
When you assign an HTTP status code of 200 to a redirect rule, it becomes a rewrite . This means that the URL in the visitor’s address bar remains the same, while Netlify’s servers fetch the new location behind the scenes.
Hi, @shafiemukhre. That site (www.parent.com) isn’t hosted at Netlify. That site is hosted at Shopify. Only sites hosted at Netlify can use our redirects feature.
Thanks @coelmay. I see, there’s multiple domain in the headers. How does that effect search engine ability to crawl?
If anyone can share articles/blogs or readings materials on how exactly redirect/rewrite affect SEO, that will be great. If it is specific to Netlify, that’s even better. I am quite loss at this point.
My current understanding is that the Search Engine crawler bot will open a page and crawl all available links available on that page equivalent to someone clicking it and go to next page. As long as the next link on that page is using the same domain, the bot will crawl that next page. If I remember correctly, I learned this from a video on youtube from an Engineer at Google that’s working on Search Engine.
Therefore, my understanding is that, it doesn’t matter if there’s redirect or multiple redirects, as long the final destination of the redirect happened to be on the same domain, the bot will keep crawling it and indexs it.
It probably doesn’t affect the ability to crawl. I am not a Google SEO expert (they likely exist on Google SEO specific forums) and have little understanding how Google “actually” works in this regard, but perhaps they are saying it is not available because of these headers. Possible?
There is also the fact that in your screenshot it shows a 5xx error but doesn’t specify which one.
You say Bing also doesn’t register the URL. What is the reasoning they give?
I got the screenshot of the 5xx error from Google Search console. I am not sure if Bing have something similar or not, will have a look and give a try.
I made an observation that Bing is not indexing the urls as well because of the same result I got when I did the search on Bing.
Answer: That’ll depend on the headers that get set by the external source This little tidbit might be handy for improving SEO for proxied content (consider setting a canonical header pointing to the proxied path on dremio.com).
From my personal experience in getting Google to index my website, I get 5xx errors from time to time even when not having any redirect. It’s a simple static page, no JS based rendering, nothing and yet I get the error. However, I haven’t got one in a few months now.
I tend to ignore them and they automatically get fixed when Google rechecks my website. But yes, I ignore them because my websites are more like personal stuff, so not highly SEO critical for me. But if you’re seeing the errors frequently, I think it’s worth asking in some SEO forums too, just to be sure of the possible causes.
Thank you @hrishikesh for your input! It’s good to know that other people faced this issue as well.
When I created a new content, I noticed that it may take 7 - 10 days for Google to index it. So, I am expecting the same amount of time. Now after 2 weeks, still nothing happened. This is a new implementation though.
Yeah, this is a bit SEO critical to us. It’s fine to wait if it’s going to take longer time, we just need to confirm and know whether we are doing the right implementation or not.
Thanks for the suggestion, will ask some other SEO forums and SEO agency on this matter as well in parallel.
I’ve replied to the helpdesk ticket too, but adding a part of the reply here.
From the internal logs, I was not able to find Googlebot User Agent at least in the last 7 days. Furthermore, Bing bot was able to get a 200. If it’s still not indexing your website, I think it’s a tough case.
Thank you @hrishikesh! It seems that this issue was automatically fixed after about a month without us doing anything. I’m assuming that’s because it’s either they had to re-crawl the pages to notice it or it just takes more time.
