Hi, I’m building a website with a lots of pages and serverless functions — there’s currently around 10,000 pages — and each page has a serverless function it is paired with.
During the deploy stage I get an error saying: Execution timed out after 26m59.304740045s and Failing build: Failed to build site.
However, after it has given me this error, it changes its mind and the site successfully goes live.
The site build and function bundling is very fast — but the deploy stage is very slow — I’m planning on making the site much larger and am looking for a solution to this problem in advance.
Why is it throwing this error, then successfully launching the site?
How can I improve the speed of the deploy stage?
Any help would be greatly appreciated.
Here are the relevant logs from the build:
(build.command completed in 32.1s)
(Functions bundling completed in 1m 32.3s)
11:20:57 PM: 3. Deploy site
11:20:57 PM: ────────────────────────────────────────────────────────────────
11:44:58 PM: Section completed: deploying
11:44:58 PM: Site deploy was successfully initiated
11:44:58 PM: (Deploy site completed in 24m 44.8s)
11:44:58 PM: Netlify Build Complete
11:44:58 PM: ────────────────────────────────────────────────────────────────
11:44:58 PM: (Netlify Build completed in 26m 49.4s)
11:44:58 PM: Execution timed out after 26m59.304740045s
11:44:58 PM: Error running command: Command did not finish within the time limit
11:44:58 PM: Failed during stage 'building site': Command did not finish within the time limit
11:44:58 PM: Failing build: Failed to build site
11:44:59 PM: Finished processing build request in 27m2.479s
11:45:15 PM: Starting post processing
11:45:15 PM: Post processing - HTML
11:49:28 PM: Post processing - header rules
11:49:28 PM: Post processing - redirect rules
11:49:28 PM: Post processing done
11:49:29 PM: Section completed: postprocessing
11:49:30 PM: Site is live ✨
Welcome to the forums and thanks so much for reaching out!
Files over 10 MB in size are not well-supported by our CDN and may fail to upload to our system, causing your entire deploy to fail. You should host large content elsewhere, such as YouTube embedded videos.
Sites with tens of thousands of html files can lead to long processing times. This shouldn’t cause the deploy to fail, but even a “quick” manual deploy can take quite awhile (many minutes) to finish if you have tens of thousands of files.
Yes, there are no limits (as you can see, we haven’t blocked you on the number of functions anywhere), I was just curious to know what was so intensive that needed 14k+ functions. I thought maybe we can discuss and find a way to optimise it for you as well as us, because if you keep on increasing the number, the build time will continue increasing.
Based on your description, I’m assuming you’re deploying a separate function to handle each and every task (conversion). If that’s the case, I think it’s okay to have an individual function.
I’ve bumped the build time limit to 2 hours, so I don’t think you should be running into this issue any time soon.
Thanks for the quick response, and thank you for increasing the time — at the current rate I’m building the site it will have around 200k functions very soon. Which would take about 10 hours to deploy — which is too long even if it is possible.
I’ve done a lot to optimize the build — the site only takes about 1 minute to actually build — whereas the deploy stage seems to be out of my hands.
Is there a way of optimizing the deploy stage — other than having a smaller site, and fewer functions?
Hi Sam, thanks for the links, but they seem to be about optimizing build time — which isn’t really the issue here. I really want to get to the bottom of this.
The site builds in 1 minute — including the functions.
That is to say, the code I wrote to actually build all the files, including the .js files in the /functions/ directory, builds in 1 minute.
It then takes more than an hour to deploy.
If I remove the serverless functions from my site, the site takes 1 minute to build and deploy — and i know I can build the serverless functions in that same minute — i can even do it on my laptop.
So, it’s Netlify deploying the serverless functions that takes an hour — I can optimize my build as much as possible, but I have no control over that part of the deploy. I can only control the speed that my code builds the static assets that Netlify uses to build the site.
I don’t really understand what is materially different about a serverless function and a static page, that makes these serverless functions take 200 times longer than the page to deploy — and I’m sure I must be missing something here.
I have come up with a less than ideal way of getting around this issue, that will work for my specific project (at least for now) — which will mean I’m using a lot less serverless functions.
But I’d still really appreciate understanding what is going on here. If I carried on building with that many functions I would exceed my monthly build minutes — which is frustrating since it’s not really in my control, and prevents me from making my project as userfriendly as I’d like.
We discussed this with the devs and there could be some sad news coming along. While we have no immediate plans to add system-level restrictions to upload so many functions per site, the team has decided upon 1000 as a “safe number” of functions per site. You can still use anything above that, but as use case like yours, where you need 200k functions probably cannot be supported for a long term.
I can understand you’re already at 14k+ functions and thus, too deep into your project’s development pipeline to even think about refactoring anything, but here is my suggestion. If you can somehow refactor your project to reduce the number of functions, that would be great. I’d advise using something like Express/Fastify to create a single function bundle while keeping all your routes separate. See here for example:
In spite of having multiple routes, that ends up as a single function bundle. So, you can probably still maintain your project structure, while adding something like dynamic routing parameters, etc. inside Express/Fastify which would help us support your site further. Not to mention, it will also cut down on build times, so you can continue to have faster builds.
Hi @hrishikesh, thanks for all your help.
I’ve found a solution, all the tools like grams to ounces converter when the user has no-js they get forwarded to the weight converter tool with grams and oucnes prefilled — same for all the tools that have a subset of the functionality of a different tool.
That significantly reduces the number of functions down to 29 for now — I can’t imagine with this new system the number exceeding 1000.