[Support Guide] Understanding and debugging prerendering

Hi, @Angel1. There is no way to clear the prerender cache at this time. Quoting the documentation on this:

Our built-in prerendering service will cache prerendered pages for between 24 and 48 hours; this is not adjustable.

Redeploying will clear the CDN cache but not the prerender cache. The only way to clear the prerender cache is to wait.

@luke Not beeing able to redeploy is major drawback to this feature.
If I uploaded some buggy code, and Google happened to index the prerendered pages, it means I’m blocked with a bad index for up to 48h with no means clean the problem ?

Hi, @laem. You have correctly understood what is being said in this topic.

Hi @luke does prerender already support Outlook preview agent, Clickup, telegram?
If not, are you guys able to add it?

Can anyone help me understand why prerendering is working on certain pages and not others?

They are both product pages using the exact same type of code.

Not working: Data Strategy Professionals

Working: Data Strategy Professionals

Custom Helmet component:

export default function CustomHelmet({ description, name, link, img, alt }) {
  return (
    <Helmet>
      <title>{name}</title>
      <meta name="description" content={description} />
      <meta property="og:title" content={name} />
      {/* OpenGraph tags */}
      <meta name="og:url" content={"https://www.datastrategypros.com" + link} />
      <meta name="og:description" content={description} />
      <meta name="og:image" content={img} />
      <meta name="og:image:alt" content={alt} />
      <meta name="og:type" content="website" />
      {/* Twitter Card tags */}
      <meta name="twitter:title" content={name} />
      <meta name="twitter:description" content={description} />
      <meta name="twitter:image" content={img} />
      <meta name="twitter:card" content="summary" />
      <meta name="twitter:creator" content="@Nicole_Janeway" />
    </Helmet>
  );
}

Implementation (exact same in both)

      <CustomHelmet
        description={product.description}
        name={product.name}
        link={product.link}
        img={product.img}
        alt={product.alt}
      />

We’re using the all the product values in other places on the product page, so that’s not the problem.

Is this enabled for discourse forums? I’m not sure what their user agent is–it kind of looks like it might just be Mozilla?

eg., this page is rendered by netlify’s prerender and it works on places like discord and whatsapp

https://mirlo.space/widget/track/142

If you find the documented user agent for Discourse we can check and add that if it doesn’t exist.

It looks like discourse has a bunch of special handlers for specific websites and it’s unclear to me what the user agent is when it fetches information.

Looking at this thread, it actually looks like it could be as defined in this file

Is it possible to include a partial user agent string? Discourse Forum Onebox v#{Discourse::VERSION::STRING} so something like Discourse Forum Onebox?

Our devs would only consider adding if it comes from their official documentation or a publicly acknowledged post. The link you mentioned doesn’t seem to be the official one.

That’s their source code and their support forum. I’m not sure how much more official it’ll get than that.

Sorry, I missed the part in the codebase. I have added a request for the devs to review.

1 Like

This should be rolled out.

1 Like

Thanks @hrishikesh

We’ve got another request. Noticed that our cards weren’t rendering correctly on Mastodon, and so I started looking through the Mastodon code to find a user agent.

Found this: mastodon/app/services/fetch_link_card_service.rb at main · mastodon/mastodon · GitHub which I think is where the preview gets fetched. This references the user_agent string here mastodon/lib/mastodon/version.rb at a2f02a07758c32f0dcc6388b4f30ca5a84e762f3 · mastodon/mastodon · GitHub

Which generates a user agent which looks like this Mastodon Bot - WhatMyUserAgent.com

So I think the user agent string to add should be “Mastodon”

Also want to point out that our dev @psi can’t add things to this thread because there’s a rule around not being allowed to post more than 3 times in a thread. That’s a pain.

1 Like

Bumping this, hopefully someone can look into it?

1 Like

Unfortunately, that user agent looks something like: http.rb/5.1.1 (Mastodon/4.3.0-nightly.2024-02-03; +https://mastodon.online/).

Since we match from the start of the string, we would have to match http.rb and I’m not sure if that would be the intended behaviour, but I’ll try asking.

1 Like