Prerender: setting `window.prerenderReady = true` before all http requests done

Support
1

I need to make a short http request before taking the prerender snapshot. It seems like setting window.prerenderReady = false in the html, then to true in js after my request should do the trick. However, the page has a few other long-running requests that I don’t care about for the prerender. Even after setting window.prerenderReady = true, the service is waiting for all other requests to complete before prerendering. This causes many scrapers (including Facebook’s) to time out.

This seems like the same issue described here by @tommedema in the in the netlify/prerender repo.

As Tom noted, the issue seems to be with this line:
lib/browsers/chrome.js:496

resolve((!shouldWaitForPrerenderReady && doneLoading) || (shouldWaitForPrerenderReady && doneLoading && prerenderReady));

Which will resolve to false if !doneLoading, even if shouldWaitForPrerenderReady && prerenderReady.

Is there any workaround such that the snapshot can be triggered as soon as window.prerenderReady = true, regardless of any other in-flight requests?

I built a short example that illustrates the issue:

I have one site that makes a 3s network request, and then sets window.prerenderReady = true. Testing with the prerender cli shows that this completes successfully in ~5s.

3s request

2020-09-03T14:58:11.893Z getting https://5f5104670699ff348c2fc8d9--netlify-prerender-test.netlify.app/async
2020-09-03T14:58:17.526Z got 200 in 5633ms for https://5f5104670699ff348c2fc8d9--netlify-prerender-test.netlify.app/async

Another site is the exact same, except it also kicks off an unrelated 20s request. The 3s request returns and sets window.prerenderReady = true, but the prerendering service continues to wait for the 20s request to complete, which results in a timeout after ~20s.

3s request with 20s request

2020-09-03T15:02:16.183Z getting https://5f51055379c6fa38d48d4131--netlify-prerender-test.netlify.app/async
2020-09-03T15:02:36.282Z page timed out https://5f51055379c6fa38d48d4131--netlify-prerender-test.netlify.app/async
2020-09-03T15:02:36.288Z got 200 in 20105ms for https://5f51055379c6fa38d48d4131--netlify-prerender-test.netlify.app/async
2

One thought on a potential workaround – is there a way in code to know when netlify is prerendering so that I can not trigger long-running requests? (this is a bad option because it will require wrapping a bunch of requests in extra logic, but better than nothing)

3

While I don’t think you can affect the status of doneloading, perhaps you can affect its prerequisites such as the one mentioned here?

I don’t know that that is the best path, but it might be worth looking into.

If that isn’t helpful, you can use something like this in your client-side code (which is also parsed/used by the prerender service - I’m sure you knew that, just explaining for future visitors to this thread who may not understand the situation as well as you do), to know if your content is being served to a prerender service:

const isPrerender = /HeadlessChrome/.test(window.navigator.userAgent)
4

Hi @fool, thanks for the response. Could you elaborate more on what you mean by “Affecting the prerequisites”? Not sure I understand.

Thanks for the isPrerender check. I’ve been searching for something like that. One main concern though: if I use isPrerender to render something different (ie not serve the content that takes a long time to load) will search engines consider it cloaking and downrank us?

5

I mean “can you affect what domContentEventFired behaves like, to change this behavior based on the existing logic”? I have no idea if that is possible or sensible, but it looks like something that might be addressable from your JS code.

I cannot speak to what search engines think or do. I can speak to some other fairly high profile customers who rely on this (use case I can recall: the Zendesk helpdesk “chat” widget breaks prerendering, so they just skip its initialization in prerendered requests. The content is otherwise identical).