[Support Guide] Understanding and debugging prerendering

Support Support Guides
, ,
95

Hi @patrickstox,

I’d advise you to watch this thread:

We’d post an update there as soon as this works out.

1 Like
99

I have no way of confirming this, but due to a massive increase in bandwidth usage I believe prerendering is counted against bandwidth usage. Can you please check and confirm if this is intentional or a bug.

100

Hi, @CaffeDoppio. The email address you used for this forum isn’t associated with any sites at Netlify. In order to research this, we will need to know which site you are asking about.

You can post that information publicly or you can private message (PM) that to one of our support staff. I’ve confirmed that PMs are enabled for your forum login. Please keep in mind that only one person can see the PM and this will likely mean a slower reply than posting the information publicly. Please feel free to reply to however you prefer though.

101

Hi @luke thanks for your reply and for looking into this. It’s ccgtrader.net.

103

I have prerendering working with Twitter / Facebook, but for some reason it doesnt seem to work with opengraph.xyz can anyone point me in the right direction?

The website I am testing is https://sharehomethegame.com/characters

You can use any of the characters under this page to test different URLs.

104

Is that some service? If so, it might have a customer pre-render agent that we might not have configured on our end. If you can find its user agent and let us know, we can investigate if it can be added.

106

Hi, we will like to deactivate the External Prerender for our site prod-drinks-consumer-marketplace.netlify.app and switch to the internal pretender service by Netlify?

107

Hey @Spazwik,

This has been updated.

108

Thank you, will test it out to see if it works properly.

109

Hi all, really interested in using this for a website that needs to be SEO friendly and have some questions.

  1. This has been in Beta for awhile. Is it expected to be supported long term, or is there any hint that it might get dropped in the future?
  2. Docs say it may not be available on all plans. Are there differences in plans?
  3. How does Netlify crawl my site to know what routes to prerender?
110

Hi @Cam8433,

  1. While the feature is in Beta, there are no plans to drop support any time soon.

  2. Configuring external pre-redendering is something that’s not available on the starter plan, but Netlify built-in pre-rendering is available for all plans.

  3. Netlify will not pre-render your site unless someone requests a page. And by someone, I mean a pre-render agent. Once Googlebot for example requests a page of your website, Netlify will pre-render it and store it in a cache.

1 Like
112

I’m sorry if this has already been asked in the thread, but given the amount of attention this topic seems to garner over the years, I was wondering if the User Agent list that’s Netlify’s Prerenderer is using could be made open-source somehow… you could review new PRs to it and have a trace of all changes (not to mention the actual list itself) for people to be able to see. I myself am wondering if the Android/iOS messaging apps’ user agents are in there. I have prerendering enabled and my og data isn’t showing up in Google Chat (as it does for other sites - static/SSR presumably)

113

Do you know the agent of Google Chat?

114

Hi @hrishikesh! I’m not exactly sure, but I figure it’s one of those listed in their crawlers index. I’d like to do a visual diff.

115

here’s the current list. Sorry we can’t publish it more proactively, but you can always ask in this thread. It does not change very often these days. This is a regular expression looking for substrings in user agents, case-insensitively.

baiduspider|twitterbot|facebookexternalhit|facebot|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|SocialFlow|Net::Curl::Simple|Snipcart|Googlebot|outbrain|pinterestbot|pinterest/0|slackbot|vkShare|W3C_Validator|redditbot|Mediapartners-Google|AdsBot-Google|parsely|DuckDuckBot|whatsapp|Hatena|Screaming Frog SEO Spider|bingbot|Sajaribot|DashLinkPreviews|Discordbot|RankSonicBot|lyticsbot|YandexBot/|YandexWebmaster/|naytev-url-scraper|newspicksbot/|Swiftbot/|mattermost|Applebot/|snapchat|viber|proximic|iframely/|upday|Google Web Preview|AhrefsBot/|AhrefsSiteAudit/|GoogleSites|PetalBot

1 Like
117

Thank you @fool. It looks like you’ve got most of Google’s bots covered. Not sure why it’s not working in Google Chat, but it helps to have the latest list!

118

Is there not any way to bust the cache? I assume re-deploying would bust the cache but it seems to be serving from the previously prerendered page regardless.