Deploy stage takes a long time, throws an error, but then works anyway

Hi, I’m building a website with a lots of pages and serverless functions — there’s currently around 10,000 pages — and each page has a serverless function it is paired with.

During the deploy stage I get an error saying: Execution timed out after 26m59.304740045s and Failing build: Failed to build site.

However, after it has given me this error, it changes its mind and the site successfully goes live.

The site build and function bundling is very fast — but the deploy stage is very slow — I’m planning on making the site much larger and am looking for a solution to this problem in advance.

  • Why is it throwing this error, then successfully launching the site?
  • How can I improve the speed of the deploy stage?

Any help would be greatly appreciated.

Here are the relevant logs from the build:

(build.command completed in 32.1s)

(Functions bundling completed in 1m 32.3s)

11:20:57 PM:   3. Deploy site                                                
11:20:57 PM: ────────────────────────────────────────────────────────────────
11:20:57 PM: ​
11:44:58 PM: Section completed: deploying
11:44:58 PM: Site deploy was successfully initiated
11:44:58 PM: ​
11:44:58 PM: (Deploy site completed in 24m 44.8s)
11:44:58 PM: ​
11:44:58 PM:   Netlify Build Complete                                        
11:44:58 PM: ────────────────────────────────────────────────────────────────
11:44:58 PM: ​
11:44:58 PM: (Netlify Build completed in 26m 49.4s)
11:44:58 PM: Execution timed out after 26m59.304740045s
11:44:58 PM: Error running command: Command did not finish within the time limit
11:44:58 PM: Failed during stage 'building site': Command did not finish within the time limit
11:44:58 PM: Failing build: Failed to build site
11:44:59 PM: Finished processing build request in 27m2.479s
11:45:15 PM: Starting post processing
11:45:15 PM: Post processing - HTML
11:49:28 PM: Post processing - header rules
11:49:28 PM: Post processing - redirect rules
11:49:28 PM: Post processing done
11:49:29 PM: Section completed: postprocessing
11:49:30 PM: Site is live ✨

Hi @ShadowfaxRodeo :wave:t6:

Welcome to the forums and thanks so much for reaching out! :blush:

Files over 10 MB in size are not well-supported by our CDN and may fail to upload to our system, causing your entire deploy to fail. You should host large content elsewhere, such as YouTube embedded videos.

Sites with tens of thousands of html files can lead to long processing times. This shouldn’t cause the deploy to fail, but even a “quick” manual deploy can take quite awhile (many minutes) to finish if you have tens of thousands of files.

Here is a document that has a bit more information. Build troubleshooting tips | Netlify Docs

Hi SamO,

Thanks for the response. The site doesn’t have a long build time, it only takes a minute. However, the function bundling and deploy takes 35 minutes.

There does seem to be a time limit applied to how long a deploy takes. As the error below shows.

I’ve done all I can to make the site build as fast as possible, but I plan to make the site much larger — and so this issue will only get worse.

5:36:56 PM:   3. Deploy site                                                
5:36:56 PM: ────────────────────────────────────────────────────────────────
5:36:56 PM: ​
5:36:56 PM: Starting to deploy site from 'public'
5:36:57 PM: Calculating files to upload
5:37:11 PM: 7495 new files to upload
5:37:11 PM: 14424 new functions to upload
5:37:11 PM: 0% complete
5:37:12 PM: 5% complete
5:37:13 PM: 10% complete
5:37:14 PM: 15% complete
5:37:16 PM: 20% complete
5:37:17 PM: 25% complete
5:37:18 PM: 30% complete
5:37:19 PM: 35% complete
5:37:20 PM: 40% complete
5:37:21 PM: 45% complete
5:37:22 PM: 50% complete
5:37:23 PM: 55% complete
5:37:25 PM: 60% complete
5:37:26 PM: 65% complete
5:37:27 PM: 70% complete
5:37:28 PM: 75% complete
5:37:29 PM: 80% complete
5:37:31 PM: 85% complete
5:37:32 PM: 90% complete
5:37:33 PM: 95% complete
5:37:34 PM: 100% complete
6:00:29 PM: Section completed: deploying
6:00:29 PM: Execution timed out after 36m1.075555473s
6:00:29 PM: Error running command: Command did not finish within the time limit
6:00:29 PM: Failing build: Failed to build site
6:00:29 PM: Failed during stage 'building site': Command did not finish within the time limit
6:00:30 PM: Finished processing build request in 36m5.974s

There are no large files — only html files, and a couple of tiny static files.

If you could advise on how to avoid this timeout issue, or at least clarify what the the error is, I’d greatly appreciate it.

Thank you

While we can bump the timelimit, I’m more curious to know about your use case. Why do you need 14k functions for? What exactly are you doing?

I’m building a website with basic tools like unit converters — they each have a no JavaScript fallback for people using older devices, or with security and privacy concerns.

Before I started working on this I reached out on this forum to ask if there was a limit to the number of functions I could deploy — and you assured me that there was not.

So, if we could sort this out that would be great, but otherwise please be honest about any technical issues I will face because I will need to find another way to get this working.

As you can see, that message was sent way back in september — I’ve spent a long time working on this system. So if it isn’t possible I’d really appreciate you letting me know now.

You can take a look at the site here to get an idea of the kind of thing it does:

Yes, there are no limits (as you can see, we haven’t blocked you on the number of functions anywhere), I was just curious to know what was so intensive that needed 14k+ functions. I thought maybe we can discuss and find a way to optimise it for you as well as us, because if you keep on increasing the number, the build time will continue increasing.

Based on your description, I’m assuming you’re deploying a separate function to handle each and every task (conversion). If that’s the case, I think it’s okay to have an individual function.

I’ve bumped the build time limit to 2 hours, so I don’t think you should be running into this issue any time soon.

Thanks for the quick response, and thank you for increasing the time — at the current rate I’m building the site it will have around 200k functions very soon. Which would take about 10 hours to deploy — which is too long even if it is possible.

I’ve done a lot to optimize the build — the site only takes about 1 minute to actually build — whereas the deploy stage seems to be out of my hands.

Is there a way of optimizing the deploy stage — other than having a smaller site, and fewer functions?

Sure here are a few resource I recommend. I hope you can find your use case.

Hi Sam, thanks for the links, but they seem to be about optimizing build time — which isn’t really the issue here. I really want to get to the bottom of this.

The site builds in 1 minute — including the functions.
That is to say, the code I wrote to actually build all the files, including the .js files in the /functions/ directory, builds in 1 minute.

It then takes more than an hour to deploy.

If I remove the serverless functions from my site, the site takes 1 minute to build and deploy — and i know I can build the serverless functions in that same minute — i can even do it on my laptop.

So, it’s Netlify deploying the serverless functions that takes an hour — I can optimize my build as much as possible, but I have no control over that part of the deploy. I can only control the speed that my code builds the static assets that Netlify uses to build the site.

I don’t really understand what is materially different about a serverless function and a static page, that makes these serverless functions take 200 times longer than the page to deploy — and I’m sure I must be missing something here.

I have come up with a less than ideal way of getting around this issue, that will work for my specific project (at least for now) — which will mean I’m using a lot less serverless functions.

But I’d still really appreciate understanding what is going on here. If I carried on building with that many functions I would exceed my monthly build minutes — which is frustrating since it’s not really in my control, and prevents me from making my project as userfriendly as I’d like.

Many thanks

Hey @ShadowfaxRodeo,

We discussed this with the devs and there could be some sad news coming along. While we have no immediate plans to add system-level restrictions to upload so many functions per site, the team has decided upon 1000 as a “safe number” of functions per site. You can still use anything above that, but as use case like yours, where you need 200k functions probably cannot be supported for a long term.

I can understand you’re already at 14k+ functions and thus, too deep into your project’s development pipeline to even think about refactoring anything, but here is my suggestion. If you can somehow refactor your project to reduce the number of functions, that would be great. I’d advise using something like Express/Fastify to create a single function bundle while keeping all your routes separate. See here for example:

In spite of having multiple routes, that ends up as a single function bundle. So, you can probably still maintain your project structure, while adding something like dynamic routing parameters, etc. inside Express/Fastify which would help us support your site further. Not to mention, it will also cut down on build times, so you can continue to have faster builds.

Hi @hrishikesh, thanks for all your help.
I’ve found a solution, all the tools like grams to ounces converter when the user has no-js they get forwarded to the weight converter tool with grams and oucnes prefilled — same for all the tools that have a subset of the functionality of a different tool.

That significantly reduces the number of functions down to 29 for now — I can’t imagine with this new system the number exceeding 1000.

Thanks again,

1 Like

thank you for sharing your solution! This is definitely helpful for other users :+1:t5: