User agent for Netlify bots

Hi,

I’m getting a “headless chrome” visitor on my analytics software every time I deploy on Netlify – I’m guessing that’s a Netlify bot, possibly to screenshot the built site for the dashboard. (More details here.)

Do Netlify bots have a user-agent that could be filtered out by the analytics software?

Thank you!

Hi, @mr-islam. The most unique string I can see in the existing user-agent header is “HeadlessChrome” itself.

I can also enter a feature request asking that this snapshot service use some other, more specific string.

Note, the URL used for the screenshot always has the deploy id and the site subdomain included in it. For example:

https://5f3b5efaeea0a8b06442b086--example.netlify.app/

It might also be possible to filter the reporting based on that pattern as well.

If there are other questions and/or if we should submit the feature request for a more specific user-agent, please let us know.

Hi @luke,

I think it would be best if you submit the feature request for a more specific user-agent – then it could be added to the shared reference of bots maintained by Matomo Analytics and also used by many other analytics services.

(I think this would be a more maintainable and long-term solution than setting up a filter to exclude the URL pattern used for the screenshot.)

As for the new user-agent, it might be easier if it included the string “bot” or anything that matches the regex pattern below, since this is the pattern is used by the shared Matomo reference to filter out most bots automatically:

[a-z0-9\-_]*((?<!cu|power[ _]|m[ _])bot(?![ _]TAB|[ _]?5[0-9])|crawler|crawl|checker|archiver|transcoder|spider)([^a-z]|$)

Or if you choose a different user-agent, we can file a PR for Matomo to include it in their list explicitly

Thanks for the suggestion! It’s a good one, but we won’t be changing it since many customers have code relying on the current version and we have no good way to know who all of them are and inform them before we make a breaking change.

Sorry I don’t have better news for you today.

No problem, totally understandable!

Thank you for your help and time :blush: