Og/twitter previews not working

[wisabid]

Could you please help me find why twitter/OG previews are not working for this domain? I have everything implemented and I thought i might have to give some time for domains propagation. Iam afraid thats not the case. Twitter card validator is working fine, but i get no previews while sharing on whatsapp or anywhere else (including twitter). But it works on facebook. Please help. Also its worth noting that the domain name server has recently been transfered from zeit’s now.

URL - https://www.wismim.com

[luke]

Hi, most Open Graph (OG) card services do not run javascript so if your site requires javascript before page is properly displayed it is often helpful to enable a prerendering service for your site.

We have more about this in our documentation here:

Prerendering

The Facebook validator is showing a card but think it is using the image OG tag below to make the card:

https://firebasestorage.googleapis.com/v0/b/wismim-27d50.appspot.com/o/wismim-social.jpg?alt=media

You might test enabling prerendering to see if this helps with Twitter and other sites. If there are other questions, please let us know.

[wisabid]

Hi,

Thank you for your response. In that case, I’m confused how https://www.alfalvin.co.uk is working without any issues. Nothing specific has been done for it to work. I doubt wismim.com has to do with some security headers. Please have a look

Thanks

Abid

[luke]

Hi, @wisabid, I don’t have any way to see the difference between the card validator and the posted link at Twitter.

If you are seeing an issue with a response we are returning, I’ll need more information to troubleshoot. For example, an x-nf-request-id header for an incorrect response.

This might be an issue which will require Twitter to troubleshoot. Have you contacted Twitter’s technical support about this? If so, what did they say? If you haven’t contacted them about this, I would recommending doing so as a next step.

[stevenfabre]

Hi there — I have a similar issue with my website.

When I share the link with https on twitter, all the OG cards load as expected, when I share with http, OG cards can’t be found.

I just turned on pre-rendering, but that doesn’t seem to fix the issue either.

Thanks in advance for your help!

[fool]

yes, don’t share with http. Our service always redirects to https, so please always post your links with https

[stevenfabre]

Got it, thanks It would be nice if it worked as I’m expecting people to just type the domain without https on social media. You can consider that a feature request

[perry]

noted! If we make a change to this, we’ll definitely get back to you.

[reimertz]

I want to feature request this as well.

I think we can assume domain.tld is how the majority of users type out urls and this is how it has been working since previews was a thing. It was pretty funny because I assumed it was related to twitter and how they parsed this new tld I was using but then saw other urls working and realized it must be related to Netlify…

[fool]

Could anyone in this thread create me a site that shows the problem that you will keep prerendered with netlify’s prerender service enabled, so I can get a bug with good reproduction steps filed? It would be ideal if it’s a site you build once, and we can:

• help you disconnect the repo
• apply a custom domain so you don’t have to.

My own sites don’t seem to show this behavior, but they also aren’t great candidates for prerendering…

[simeon9696]

@reimertz

majority of users type out urls

Can you confirm if typing just the domain for other social media platforms like WhatsApp , Facebook and Instagram works?

[fool]

We’d welcome your research into this area if you’d like to find out for us. We don’t use those services so I don’t have a good test methodology for them

[icapulet]

Sorry, can you explain this request differently? I’m having the same issue and getting complaints from users when they share the apex domain without any protocol on Twitter, so I’d like to facilitate the development of the feature as much as I can.

[reimertz]

I’d like to iterate this too. We are building more and more sites for our clients and everything with Netlify is amazing but this is turning out to be a dealbreaker for us.

[luke]

Hi, @reimertz and @icapulet. If you have an example domain I would be happy to describe this issue in more detail. For this example, I’m using testmysite.io as an example.

If I make a request for just https://testmysite.io/ with the default user-agent that curl sends, then the page is not prerendered:

$ curl -vs https://testmysite.io/
*   Trying 165.227.0.164...
* TCP_NODELAY set
* Connected to testmysite.io (165.227.0.164) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=testmysite.io
*  start date: Nov 11 12:00:26 2020 GMT
*  expire date: Feb  9 12:00:26 2021 GMT
*  subjectAltName: host "testmysite.io" matched cert's "testmysite.io"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fe56400f600)
> GET / HTTP/2
> Host: testmysite.io
> User-Agent: curl/7.64.1
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 150)!
< HTTP/2 200
< cache-control: public, max-age=0, must-revalidate
< content-length: 1360
< content-type: text/html; charset=UTF-8
< date: Fri, 20 Nov 2020 14:34:51 GMT
< etag: "7a447848c1659e5d69c18599458b16db-ssl"
< strict-transport-security: max-age=31536000
< age: 400499
< server: Netlify
< x-nf-request-id: 52faad11-20f7-456f-ac92-d9037f81a3b0-2596346
<
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <title>Website Speed Test Tool - Testmysite.io by Netlify</title>
  <meta name="description" content="Is your site as fast and secure as it could be? Testmysite.io rates your site based on load times, performance, and security settings. Get your website's speed test results!">
  <meta name="robots" content="index, follow">
  <link rel="shortcut icon" href="favicon.ico">
  <link rel="stylesheet" href="/styles.css">
  <script> window.prerenderReady = false; </script>
</head>
<body>
  <div id="app" class="container"></div>
  <script src="/app.bundle.js"></script>
  <div class="footer">
    <a href="https://www.netlify.com/">
      <img src="https://www.netlify.com/img/global/badges/netlify-color-bg.svg" />
    </a>
  </div>
<script>
  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
  })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');

  ga('create', 'UA-42258181-10', 'auto');
  ga('send', 'pageview');

</script></body>
</html>
* Connection #0 to host testmysite.io left intact
* Closing connection 0

If I change the user-agent to “Twitterbot”, it is prerendered:

$ curl -vs -A Twitterbot https://testmysite.io/
*   Trying 165.227.12.111...
* TCP_NODELAY set
* Connected to testmysite.io (165.227.12.111) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=testmysite.io
*  start date: Nov 11 12:00:26 2020 GMT
*  expire date: Feb  9 12:00:26 2021 GMT
*  subjectAltName: host "testmysite.io" matched cert's "testmysite.io"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fa1c880f600)
> GET / HTTP/2
> Host: testmysite.io
> User-Agent: Twitterbot
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 150)!
< HTTP/2 200
< content-type: text/html;charset=UTF-8
< date: Wed, 25 Nov 2020 05:51:02 GMT
< etag: W/"772-/ApNc54+OzHvjF6ziSherETS14E"
< content-length: 1906
< age: 0
< server: Netlify
< vary: Accept-Encoding
< x-nf-request-id: ea4a4aee-9753-4a46-9073-05e2233caa8d-5794462
<
<!DOCTYPE html><html lang="en"><head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <title>Website Speed Test Tool - Testmysite.io by Netlify</title>
  <meta name="description" content="Is your site as fast and secure as it could be? Testmysite.io rates your site based on load times, performance, and security settings. Get your website's speed test results!">
  <meta name="robots" content="index, follow">
  <link rel="shortcut icon" href="favicon.ico">
  <link rel="stylesheet" href="/styles.css">

</head>
<body>
  <div id="app" class="container"><div data-reactroot="" class="app is-step-0"><form class="search"><div class="search__title"><img class="pen-arrow left" src="/b22715183c068b0120ae96598b621657.svg"><!-- react-text: 5 -->Test Your Site Here<!-- /react-text --><img class="pen-arrow right" src="/b22715183c068b0120ae96598b621657.svg"></div><input type="text" class="search__input" placeholder="Enter URL here..." value=""><div class="search__hook">Is your site as fast and secure as it could be?</div><div class="search__intro"><p class="intro-text-paragraph"><strong>Testmysite.io rates your site based on its initial global load times and its security settings.</strong></p><p class="intro-text-paragraph">Even a very optimized site made on a legacy stack like Wordpress can often become several seconds faster by being built in a modern way and served via a service such as Netlify. By using HTTPS and HTTP2 you can improve both speed and search ranking.</p></div></form><div class="app__steps"><span class="app__steps__transitions"><div class="app__steps__item"></div></span></div></div></div>

  <div class="footer">
    <a href="https://www.netlify.com/">
      <img src="https://www.netlify.com/img/global/badges/netlify-color-bg.svg">
    </a>
  </div>


* Connection #0 to host testmysite.io left intact
</body></html>* Closing connection 0

However, if I make the same two requests (with and without the Twitterbot user-agent) but this time without SSL (http:// instead of https://), then something different happens:

For the non-prerendered version (without the special user-agent), a 301 redirect occurs:

$ curl -vs http://testmysite.io/
*   Trying 138.68.244.143...
* TCP_NODELAY set
* Connected to testmysite.io (138.68.244.143) port 80 (#0)
> GET / HTTP/1.1
> Host: testmysite.io
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Cache-Control: public, max-age=0, must-revalidate
< Content-Length: 38
< Content-Type: text/plain
< Date: Mon, 23 Nov 2020 16:17:05 GMT
< Location: https://testmysite.io/
< Age: 135467
< Connection: keep-alive
< Server: Netlify
< X-NF-Request-ID: 227e1b02-1217-4b5c-a2a3-2a4e0152d036-22772393
<
Redirecting to https://testmysite.io/
* Connection #0 to host testmysite.io left intact
* Closing connection 0

However, if prerendering is enabled and a user-agent which triggers prerendering is used, then a 307 redirect happens and there is HTML returned:

$ curl -vs -A Twitterbot http://testmysite.io/
*   Trying 167.172.221.254...
* TCP_NODELAY set
* Connected to testmysite.io (167.172.221.254) port 80 (#0)
> GET / HTTP/1.1
> Host: testmysite.io
> User-Agent: Twitterbot
> Accept: */*
>
< HTTP/1.1 307 Temporary Redirect
< Content-Length: 39
< Content-Type: text/html; charset=utf-8
< Date: Wed, 25 Nov 2020 05:56:05 GMT
< Etag: W/"27-ghawzGh2y9RPAcFY59/zgzzszUE"
< Location: https://testmysite.io/
< Age: 0
< Connection: keep-alive
< Server: Netlify
< Vary: Accept-Encoding
< X-NF-Request-ID: a76337e4-f65b-440c-bb78-270818e94b6a-8283499
<
* Connection #0 to host testmysite.io left intact
<html><head></head><body></body></html>* Closing connection 0

You can see the HTML is returned when a 301 redirect without HTML would be preferrable.

We have an issue filed to change this but the behavior hasn’t been changed at this time.

The only workaround for now is to not type in the apex domain alone and to manually include https:// when typing the URL. Similar, if these services would default to HTTPS (instead of HTTP only) when the apex domain is used alone, that would also resolve the issue.

If/when this issue is known to be resolved on our side we will post an update here to let you know.

[reimertz]

Thanks for the reply.

@luke

The only workaround for now is to not type in the apex domain alone and to manually include https:// when typing the URL. Similar, if these services would default to HTTPS (instead of HTTP only) when the apex domain is used alone, that would also resolve the issue.

And sadly, this is what is turning out to be a deal breaker for us… :\

Users of social networks have been conditioned over a decade now not to having to prepend https:// and sadly, and our clients are noticing sites we build for them are not getting previewed properly when shared causing a lot of concerns and we’re at the point we might have to move to another hosting provider…

This suck so bad because EVERYTHING else with Netlify is amazing.

[reimertz]

Maybe you can make the prerender service follow the redirect and render that page instead?

[luke]

Hi, @reimertz. As I’ve said, there is an issue filed for this and we’ll post an update here if/when this changes.

[sfunfocus]

Hi, I just wanted to add that this is similarly becoming a dealbreaker for us too with clients observing that Twitter shares aren’t loading properly.

Also, weirdly enough, none of the mentioned workarounds in this thread (including https://) made any difference for us.

We’ve filed a similar bug with Twitter dev relations! thank you!

[luke]

Hi, @sfunfocus. We can take a look to see if something has changed. Does the URL work in the Twitter OG validator here?

https://cards-dev.twitter.com/validator

If it does not, what is the URL being tested?

If it does work above, however, it should work on Twitter itself as well.