Just linking in this thread, which I believe is related, (sorry to those that disagree):
It’s odd that this site, which is predominantly text (but does contain a searchindex.json @ 3.8mb) be repeatedly consumed by a Tiktok affiliated bot to the tune of 67gb of usage (approx 16,000 requests), and that the answer from Netlify support be:
If you do not wish this bot to visit your site, I would recommend programming an Edge Function to block traffic by User Agent. You can see examples of them here to give you some ideas: https://edge-functions-examples.netlify.app/
I can see how Netlify as a predominantly self-service “code your own solution” platform might propose a “do it yourself” approach, but it’s a little weird to push the onus of implementing custom abuse prevention onto the customer, especially when Netlify do promote various built in protections, that one might assume could handle it.
Hopefully whatever solution is currently in the works pertaining to this feature request is also able to take these kinds of occurrences into account.