[Support Guide] Understanding and debugging prerendering

It looks like the team blind crawler does not get the prerendered page. I attached a link: https://www.factionfinancial.com/prediction/0508a90d-010e-4344-aa83-6988ae153973 Link gets pre-rendered properly on twitter and facebook but not on team blind.

Hi, @3mp3ri0r. I’m not sure what you are asking. There is no question in your last reply.

I requested the prerendered version of that URL with the following command locally:

curl -svo prerender-index.html -H "user-agent: twitterbot" \
 https://www.krismaperkasa.com/motor-elektrik/product/alliance-gear-helical-and-bevel-gearbox-AK-series

This is the title of the response:

<title>Alliance gear helical and bevel gearbox AK series | Krisma Perkasa Engineering</title>

So, that title does appear to be prerendering correctly when I test.

@jude234, our prerendering service only triggers when a user-agent matches the list for the prerendering service. We can add the user agent that the team blind crawler uses. However, before we can do so we need to know what that user agent is. I’ve tried finding that information but I couldn’t locate it. Do you know what user-agent header they send?

@luke I unfortunately do not. I tried to check their forums. I am guessing I should reach out to their dev team?

Hi, @jude234. Yes, contacting their developers sounds like a good way to get that information.

You can also ask them to contact our support team here at Netlify and we will work to enable prerendering for their service’s HTTP requests.

Any chance we can the Embedly crawler to the list?

“Mozilla/5.0 (compatible; Embedly/0.2; +http://support.embed.ly/)”.

Thanks!

I have an interesting issue with pre-rendering not working.

Here is the issue I’ve created yesterday.

To summarize here, pre-rendering seems to be broken with any url ending with a path-param containing a dot (“.”). I’m unable to reproduce locally using Netlify’s fork of prerender.

More details can be found in the issue link above.

Please help.

Hey there, @david_opbase :wave: I have looped in the Support team. Please hang tight! We will follow up on the thread you created yesterday :slight_smile:

Hey @fool, would you mind adding a couple bots from Ahrefs to the prerender list? Thank you for your time. Bot info.

  • Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)
  • Mozilla/5.0 (compatible; AhrefsSiteAudit/6.1; +http://ahrefs.com/robot/)

Hi @patrickstox,

I’d advise you to watch this thread:

We’d post an update there as soon as this works out.

1 Like

I have no way of confirming this, but due to a massive increase in bandwidth usage I believe prerendering is counted against bandwidth usage. Can you please check and confirm if this is intentional or a bug.

Hi, @CaffeDoppio. The email address you used for this forum isn’t associated with any sites at Netlify. In order to research this, we will need to know which site you are asking about.

You can post that information publicly or you can private message (PM) that to one of our support staff. I’ve confirmed that PMs are enabled for your forum login. Please keep in mind that only one person can see the PM and this will likely mean a slower reply than posting the information publicly. Please feel free to reply to however you prefer though.

Hi @luke thanks for your reply and for looking into this. It’s ccgtrader.net.

I have prerendering working with Twitter / Facebook, but for some reason it doesnt seem to work with opengraph.xyz can anyone point me in the right direction?

The website I am testing is https://sharehomethegame.com/characters

You can use any of the characters under this page to test different URLs.

Is that some service? If so, it might have a customer pre-render agent that we might not have configured on our end. If you can find its user agent and let us know, we can investigate if it can be added.

Hi, we will like to deactivate the External Prerender for our site prod-drinks-consumer-marketplace.netlify.app and switch to the internal pretender service by Netlify?

Hey @Spazwik,

This has been updated.