[Support Guide] Understanding and debugging prerendering

Hi @Cam8433,

  1. While the feature is in Beta, there are no plans to drop support any time soon.

  2. Configuring external pre-redendering is something that’s not available on the starter plan, but Netlify built-in pre-rendering is available for all plans.

  3. Netlify will not pre-render your site unless someone requests a page. And by someone, I mean a pre-render agent. Once Googlebot for example requests a page of your website, Netlify will pre-render it and store it in a cache.

1 Like

I’m sorry if this has already been asked in the thread, but given the amount of attention this topic seems to garner over the years, I was wondering if the User Agent list that’s Netlify’s Prerenderer is using could be made open-source somehow… you could review new PRs to it and have a trace of all changes (not to mention the actual list itself) for people to be able to see. I myself am wondering if the Android/iOS messaging apps’ user agents are in there. I have prerendering enabled and my og data isn’t showing up in Google Chat (as it does for other sites - static/SSR presumably)

Do you know the agent of Google Chat?

Hi @hrishikesh! I’m not exactly sure, but I figure it’s one of those listed in their crawlers index. I’d like to do a visual diff.

here’s the current list. Sorry we can’t publish it more proactively, but you can always ask in this thread. It does not change very often these days. This is a regular expression looking for substrings in user agents, case-insensitively.

baiduspider|twitterbot|facebookexternalhit|facebot|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|SocialFlow|Net::Curl::Simple|Snipcart|Googlebot|outbrain|pinterestbot|pinterest/0|slackbot|vkShare|W3C_Validator|redditbot|Mediapartners-Google|AdsBot-Google|parsely|DuckDuckBot|whatsapp|Hatena|Screaming Frog SEO Spider|bingbot|Sajaribot|DashLinkPreviews|Discordbot|RankSonicBot|lyticsbot|YandexBot/|YandexWebmaster/|naytev-url-scraper|newspicksbot/|Swiftbot/|mattermost|Applebot/|snapchat|viber|proximic|iframely/|upday|Google Web Preview|AhrefsBot/|AhrefsSiteAudit/|GoogleSites|PetalBot

1 Like

Thank you @fool. It looks like you’ve got most of Google’s bots covered. Not sure why it’s not working in Google Chat, but it helps to have the latest list!

Is there not any way to bust the cache? I assume re-deploying would bust the cache but it seems to be serving from the previously prerendered page regardless.

Hi, @Angel1. There is no way to clear the prerender cache at this time. Quoting the documentation on this:

Our built-in prerendering service will cache prerendered pages for between 24 and 48 hours; this is not adjustable.

Redeploying will clear the CDN cache but not the prerender cache. The only way to clear the prerender cache is to wait.

@luke Not beeing able to redeploy is major drawback to this feature.
If I uploaded some buggy code, and Google happened to index the prerendered pages, it means I’m blocked with a bad index for up to 48h with no means clean the problem ?

Hi, @laem. You have correctly understood what is being said in this topic.

Hi @luke does prerender already support Outlook preview agent, Clickup, telegram?
If not, are you guys able to add it?

Can anyone help me understand why prerendering is working on certain pages and not others?

They are both product pages using the exact same type of code.

Not working: Data Strategy Professionals

Working: Data Strategy Professionals

Custom Helmet component:

export default function CustomHelmet({ description, name, link, img, alt }) {
  return (
    <Helmet>
      <title>{name}</title>
      <meta name="description" content={description} />
      <meta property="og:title" content={name} />
      {/* OpenGraph tags */}
      <meta name="og:url" content={"https://www.datastrategypros.com" + link} />
      <meta name="og:description" content={description} />
      <meta name="og:image" content={img} />
      <meta name="og:image:alt" content={alt} />
      <meta name="og:type" content="website" />
      {/* Twitter Card tags */}
      <meta name="twitter:title" content={name} />
      <meta name="twitter:description" content={description} />
      <meta name="twitter:image" content={img} />
      <meta name="twitter:card" content="summary" />
      <meta name="twitter:creator" content="@Nicole_Janeway" />
    </Helmet>
  );
}

Implementation (exact same in both)

      <CustomHelmet
        description={product.description}
        name={product.name}
        link={product.link}
        img={product.img}
        alt={product.alt}
      />

We’re using the all the product values in other places on the product page, so that’s not the problem.

Is this enabled for discourse forums? I’m not sure what their user agent is–it kind of looks like it might just be Mozilla?

eg., this page is rendered by netlify’s prerender and it works on places like discord and whatsapp

https://mirlo.space/widget/track/142

If you find the documented user agent for Discourse we can check and add that if it doesn’t exist.