Hi, @justingrant, @Ultra, and @coelmay. I just tested the site with the tool https://www.redirect-checker.org/index.php and was surprised to find it providing false information.
EDIT: The page at https://www.redirect-checker.org/index.php is giving correct information and it was our logging, not theirs, that had false information. The page at https://www.redirect-checker.org/index.php is trustworthy and I’m sorry for saying otherwise. I was wrong.
For example, it showed this:
>>> https://www.justingrant.net/
> --------------------------------------------
> 301 Moved Permanently
> --------------------------------------------
Status: 301 Moved Permanently
Code: 301
cache-control: public, max-age=0, must-revalidate
content-length: 44
content-type: text/plain
date: Tue, 18 May 2021 08:28:24 GMT
x-nf-request-id: 7830a9ef-f21c-4d4f-af96-c2f8efdd1a8d
Location: https://www.justingrant.net/
server: Netlify
age: 81192
This shows that the request was made to the URL https://www.justingrant.net/. However, we can see the requests logged at Netlify and the request is actually being made to http://www.justingrant.net/. The tool says it used HTTPS but I can see that it did not do so in reality.
So, this isn’t a redirect loop. That tool is repeatedly requesting the HTTP version 19 times and being directed to HTTPS all 19 times. That isn’t a loop but the tool incorrectly states that it is. So, I won’t be able to use that tool for future testing because I know it isn’t providing accurate data.
At Netlify, we cannot see what is happening inside of Google’s tools or apps. We can see HTTP requests at Netlify and we log details about the request and response. This enables us to troubleshoot Netlify. We have no tools to see what happens at Google and we cannot troubleshoot what they report. However, if you help us to identify the incorrect HTTP responses, we can troubleshoot Netlify.
If you want us to address the incorrect redirects, we need to identify them or reproduce them. We normally would do this by collecting the x-nf-request-id response headers (as shown in the output above). These headers are unique to a single HTTP response and never reused.
There more information about this header here:
If that header isn’t available for any reason, please send the information it replaces (or as many of these details as possible). Those details are:
- the complete URL requested
- the IP address for the system making the request
- the IP address for the CDN node that responded
- the day and time of the request (with the timezone the time is in)
Alternatively, if you can use curl or some other CLI tool to reproduce the issue, that would be perfect. If you have such a curl example, please feel free to send that instead. For example, when I test with the Googlebot user agent, I see no issue.
First, without the Googlebot user-agent:
$ curl -svo /dev/null http://www.justingrant.net/ 2>&1 | egrep '^< '
< HTTP/1.1 301 Moved Permanently
< cache-control: public, max-age=0, must-revalidate
< content-length: 44
< content-type: text/plain
< date: Wed, 19 May 2021 07:26:34 GMT
< x-nf-request-id: ad0b7b53-209c-49ff-bc93-db0815b0f947
< location: https://www.justingrant.net/
< server: Netlify
< age: 1
<
$ curl -svo /dev/null https://www.justingrant.net/ 2>&1 | egrep '^< '
< HTTP/2 301
< cache-control: public, max-age=0, must-revalidate
< content-length: 40
< content-type: text/plain
< date: Wed, 19 May 2021 07:26:40 GMT
< strict-transport-security: max-age=31536000
< server: Netlify
< location: https://justingrant.net/
< age: 0
< x-nf-request-id: edd0269d-f7c2-453e-a64e-360343545ca1
<
$ curl -svo /dev/null https://justingrant.net/ 2>&1 | egrep '^< '
< HTTP/2 200
< cache-control: public, max-age=0, must-revalidate
< content-type: text/html; charset=UTF-8
< date: Wed, 19 May 2021 07:26:49 GMT
< etag: "a78d9f0c5950661c7b6351285791199d-ssl"
< strict-transport-security: max-age=31536000
< age: 1
< server: Netlify
< x-nf-request-id: 4ff9f826-ee62-497e-af0d-9ef8a72dca81
<
There are two redirects:
- HTTP to HTTPS at
www.justingrant.net - from
www.justingrant.nettojustingrant.netusing HTTP
The third response is a 200.
The same is true with the Googlebot user-agent:
$ curl -svo /dev/null -A Googlebot http://www.justingrant.net/ 2>&1 | egrep '^< '
< HTTP/1.1 301 Moved Permanently
< cache-control: public, max-age=0, must-revalidate
< content-length: 44
< content-type: text/plain
< date: Wed, 19 May 2021 07:29:01 GMT
< x-nf-request-id: 85a32f01-dd4e-4041-9d8f-c9861440dca8
< location: https://www.justingrant.net/
< server: Netlify
< age: 0
<
$ curl -svo /dev/null -A Googlebot https://www.justingrant.net/ 2>&1 | egrep '^< '
< HTTP/2 301
< cache-control: public, max-age=0, must-revalidate
< content-length: 40
< content-type: text/plain
< date: Wed, 19 May 2021 07:29:06 GMT
< strict-transport-security: max-age=31536000
< server: Netlify
< location: https://justingrant.net/
< age: 0
< x-nf-request-id: 70ed2479-4406-40eb-991e-96b43612a443
<
luke@macbook-luke-pdx : ~/tmp/aws-cli/aws : 2021-05-19 00:29:06 :
< HTTP/2 200
< cache-control: public, max-age=0, must-revalidate
< content-type: text/html; charset=UTF-8
< date: Wed, 19 May 2021 07:29:10 GMT
< etag: "a78d9f0c5950661c7b6351285791199d-ssl"
< strict-transport-security: max-age=31536000
< age: 0
< server: Netlify
< x-nf-request-id: 766727af-9708-487b-911a-369a45691349
<
Last but not least, if you can reproduce the issue in a browser (for example by spoofing the user-agent header there to use Googlebot’s header) then making a HAR file recording would be ideal. That recording will contain all the information required - include the x-nf-request-id headers, timestamp, and more.
To summarize, asking us to explain messages inside of Google’s tools will rarely be successful as we have zero insight into those tools. We can troubleshoot Netlify itself. In order to troubleshoot the issue, we need some way to identify the incorrect HTTP responses and the x-nf-request-id header or the information is replaces are the two most common ways this can be done.
If there are questions about any of this, please let us know.