Edit: As of 7/2 I was able to resolve this. I’d rather not say how to give the knowledge to the malicious actors, but I just had to block non humans.
I’m blowing my function limits out of the water but I don’t think I have this much real traffic and I’m getting attacked by bots.
My invocation volume I’m disputing/asking about is on the Next.js Server Handler (Lambda-based Netlify Function), and I’ve traced a meaningful share of it to identifiable automated traffic (crawlers/scrapers hitting individual content pages, plus one unrelated Cloudflare verification prober) — which you’ve now blocked at the edge — but suspect a further share is coming from scrapers rotating fake user-agent strings to evade detection, which can’t be stopped by UA-based blocking alone.
I really want to respect Netlify’s generous limits and I’m doing all I personally can to fix this. I also just turned on the prerender extension for this domain, but I’m not sure if that will help or hurt me.
Has anyone made such a site and ‘solved’ this issue? Thanks!