Redirect rules filtered by user agent to prevent social previews

:wave: I host my website justin.searls.co on Netlify, and I also syndicate its posts to various platforms like Mastodon. One annoying thing about doing this with my short-form posts is that Mastodon’s crawler will hit my links back to myself and create a web card preview from the og:* meta tags:

Here’s a guide on this topic: Stopping Mastodon From Fetching Metadata for My Notes • Robb Knight

What I’d really prefer to do is have those listed as naked URLs, which would necessitate providing a different response to this (and any other) social platform crawlers. Fortunately, they all seem to specify user agents that could filter a custom response or redirect, but as far as I can tell from this forum, this isn’t currently supported.

My naive attempt at editing _redirects before landing here looked like:

/takes/*  / 204!  User-Agent=Mastodon*

It sounds like Edge Handlers may be a thing and able to handle this? cc/ @hrishikesh

Yeah, Edge Functions could solve this. It depends on how your site is structured though. What response would you like to serve these crawlers?

Any blank 200 would suffice—whatever they each ultimately need to buzz off and not generate an OpenGraph preview.

However, if I were to put an edge function on this route, would that mean every single request to these pages would invoke that function, not just from the targeted agents? That would be slow/costly for a cached static site, wouldn’t it?

export default async function(req) {
  const ua = req.headers.get('user-agent')
  const uaList = [
    'Mastodon'
  ]

  if (!ua) {
    return
  }

  if (uaList.some(u => u.includes(ua))) {
    return new Response(null, {
      status: 204
    })
  }
}

export const config = {
  excludedPath: [
    '/static/*' // or whatever may be the path of your static assets or other paths where this is not required
  ],
  onError: 'bypass',
  path: [
    '/*' // configure it as wisely as possible to target only the required paths
  ]
}

That is correct.

Slow - not so much. Sure it will have a few milliseconds of extra time than serving a static file directly, but for the most part, it should not be highly noticable.

As for costly, it depends on how much traffic you expect for these routes. Edge Functions are fairly cheap for the most part.