I didn’t tell anyone to “shut up.” Asking to keep content on a thread focused on the topic is not “stifling discussion.”
This is a feature request directed at Netlify’s support & engineering. If others want to receive updates on this feature request, they should be able to follow this thread and receive relevant information that’s directly related. For example:
status updates
questions or clarifications on the ask
mentioning related problems to potentially expand the ask
related features or use cases that may not have been considered
If Netlify disagrees with the original feature request or asks for input/feedback from the community, then that would be the appropriate time & place to share opinions. Otherwise, Opentalk is the place for lively discussion.
But the above statements are just my opinions, which aren’t relevant to this particular feature request. So this will be my only response.
If Netlify want to offer it, that’s great, it’s theoretically a win for everyone!
Every statement they’ve made indicate that a solution to this concern is already under development, so you’re playing devil’s advocate here.
(Note: This is your only contribution to the Netlify forum so far, feel free to put your boots on the ground and help your fellow developers with their problems.)
Are you saying this to belittle me or is there some other interpretation of this that I’m missing? Anyone should be able to submit a feature request. Helping others is not a prerequisite, so telling someone to do that comes off as patronizing. I won’t be frequenting the forum if this comment is representative of what to expect from established community members.
You’re playing semantics since it suits your own position.
I agree and they can, nothing is preventing them from doing so.
Little discussed is “off topic”, although you obviously disagree.
I’m not, the theoretically alludes to the following sentence.
With a free lunch someone pays, so it’s unlikely that it would truly be a win for everyone.
You’re on a brand new user account, (perhaps a throw away account), and your only interaction with the forum so far is this thread.
This forum has an extremely low involvement from developers, discussion in this thread keeps it relevant. (Go check how many seemingly small and perfectly reasonable feature requests over the years have been quietly ignored). It’d be great if you could dedicate some time to helping other users with their problems and not just make a drive-by request and then tell the two most active non-Netlify affiliated community members not to interact with the thread because it doesn’t align with your own goals.
Just linking in this thread, which I believe is related, (sorry to those that disagree):
It’s odd that this site, which is predominantly text (but does contain a searchindex.json @ 3.8mb) be repeatedly consumed by a Tiktok affiliated bot to the tune of 67gb of usage (approx 16,000 requests), and that the answer from Netlify support be:
If you do not wish this bot to visit your site, I would recommend programming an Edge Function to block traffic by User Agent. You can see examples of them here to give you some ideas: https://edge-functions-examples.netlify.app/
I can see how Netlify as a predominantly self-service “code your own solution” platform might propose a “do it yourself” approach, but it’s a little weird to push the onus of implementing custom abuse prevention onto the customer, especially when Netlify do promote various built in protections, that one might assume could handle it.
Hopefully whatever solution is currently in the works pertaining to this feature request is also able to take these kinds of occurrences into account.
I don’t disagree about paying for bandwidth overages. It’s called a Starter plan and not a Free plan as well. Overall I’m happy with Netlify as well and have no reason to switch to Cloudflare.
However, my point is that I don’t think people should be punished and pay the bill for sudden traffic spikes of gigantic proportions, and this applies no matter what plan they’re on (even if you were on the Pro plan, you would’ve still had a $104k bill). This is why, if this is going to be something that Netlify is going to bill for, it would only be fair to give users controls to eliminate that risk which the majority are not going to be able to handle, nor is it reasonable to demand or expect customers to pay a bill this high for a short and temporary spike. It seems that you agree with this. On the other hand, billing for consistent high traffic is entirely different to sudden rare spikes and completely acceptable.
It seems Netlify understand this and have said they will review all such bills before going out, although this isn’t reassuring enough for users until they put controls in their hands, because you’re relying on nothing but words and goodwill compared to legally binding invoices and T&Cs. I would personally prefer my site goes offline past a certain spend because there’s no point in charging me or other users an amount I am never going to be able to pay, and the existence of a risk like this would instead simply make me want to be a paying customer elsewhere.
I’m not sure if it’s a regional issue, since anyone can do it as well. It doesn’t have to be a bot attack either, and your site suddenly going viral for any reason can cause a big bill. Let’s say for example, a huge influencer with millions of followers came across a blog post of yours they found useful and decided to share it. Suddenly you’re dealing with tons of traffic. Probably better safe than sorry.
That said, I have had a deeper look at Cloudflare Pages and I’m starting to consider it. I was not aware they had support for headers and redirects with redirect rules you can define just like in Netlify. They most likely also have snippet injection. They also have DNSSEC which Netlify DNS doesn’t support, although there are apparently no forms or A/B testing. If I’m not mistaken, although on CF you have unlimited bandwidth for free, you can also block entire regions if you need to.
@Nour Your position makes perfect sense, I agree it shouldn’t be up to the customer to blindly trust that Netlify will “do the right thing”, as evidenced by what @josephlouthan has experienced thus far.
While Netlify have given no indication of having done so, I’d hope the abuse of your site by the bot isn’t considered “usage” at all. It’s a contrived example, but if you have 100gb of included quota, usually use 50gb of usage, and a wild bot consumes 70gb, (entirely unchecked by Netlify’s systems), you shouldn’t get charged an overage fee. You’ve had your usual 50gb of usage, and Netlify’s systems failed to avert anomalous usage of 70gb.
There is no option to do that, or really to restrict/limit anything outside of deploying your own solution, which in turn would leverage a metered resource like Edge Functions (albeit at a lower rate than bandwidth).
Hence one of the core needs for this feature request, customers shouldn’t get dinged for anomalous usage, and with better controls they could help Netlify determine what they consider to be anomalous for their site.
Something is more likely to be announced/released regarding this feature request due to the wider attention received, but Feature Request threads in particular tend to die a silent death. If you want a given feature you should “like” the first post, and/or interact with the thread to say why it’s important to you.
Thank you. I remember this thread and posted my support for it in the past. It doesn’t seem to be planned, at least any time in the short term future, but it wasn’t a big deal because everything else Netlify offers is great.
Now I’m starting to reconsider though, especially when everything I need from Netlify seems to be offered by CF, and most importantly, because I don’t want to live in the constant fear (instead of anticipation) of my site going viral and getting hundreds of visits, which is a real possibility especially as I plan to post more which includes my own creative content in the future.
Hopefully the bandwidth issue is addressed very soon, because I’ve had an amazing experience with Netlify so far, but it’s currently seeming difficult to resist how much sense it makes when I take everything in consideration for migrating elsewhere.
Support will be reviewing all large bills BEFORE they go out.
That wasn’t the case though was it? Your policy was 20%, then 5%, payment. And even with this assurance it’s at your discretion (which in this case was initially poor)
What do Netlify get out of it?
What they’ve always got out of it. A developer deploying a site to their service that may need their paid plan for one of the many future sites, if not this one, that they develop. Ultimately a spend cap robs Netlify of 2 scenarios: profiting from illegitimate traffic, and profiting off a 1 time missed opportunity when a free tier site gets popular. Anything else can be offset by tweaking the limits.
Neither of those two should be a serious factor when considering the provision of a hosting platform, they should be making money by providing a reliable long term solution, not whacking people with unexpected bills when their situation changes. It’s gouging under the guise of “keeping the service available”. The reasonable answer to “we don’t care if it isn’t available” shouldn’t be “but then how can Netlify gouge?”
This just sounded like a bit of a shakedown to me and there is no knowing how many times it has played out especially due to Netlify’s penchant for switching to private support tickets for these kinds of occurrences.
@josephlouthan Did you end up incurring any charges from your dance with Bytespider?
It also robs them of the likely primary scenario though, which is where legitimate usage of any resource reaches a point where the user might incur an overage charge and instead decides to switch to a Pro Plan.
Let me update my situation: I went back and checked the daily bandwidth. When I blocked the robot, bandwidth did drop to 250MB/day. But the last week, it is slowly climbing back to 1GB/day.
This wouldn’t be a huge deal except my Google Analytics tells a much different tale and it feels closer to the truth:
They should be making money from providing a solid and predictable hosting platform, not through overspends based on arbitrary limits.
But yes, they might miss out on profit for say 1 day. On bandwidth they wouldn’t have provided. But they’ve gained a paying customer from a free tier. Which is exactly the gamble they were trying to make. There’s no need to gouge the market further. If it’s not a great trade off they should change their limits, not profit off of unexpected traffic, illegitimate or otherwise
Thanks for all the additional information @josephlouthan!
It does a great job of outlining how you’ve had to effectively self-solve.
Have you reached out to Netlify to see if they can provide an indication of why usage has climbed back up?
As far as I’m aware, with the Open Source plan just being the equivalent of a Pro plan, it’s entirely possible (although less likely due to the increased plan limits) that you could incur charges if the usage spirals out.
@hrishikesh is there an ETA for when this feature might be available? I love the convenience of Netlify but I’m currently advising clients to go down the VPS route until this feature exists
@hrishikesh I’m also very much looking forward to this feature. The fundamental strategic decision which hosting to recommend to our clients depends on it. Is there any update on this? The last estimate seems to have been Q3, now Q4 is coming to it’s end.
A second question: Is there any additional security in place already for customers with paid plans to prevent Horror Bills? Or are these only possible with the free plan?
