With the recent changes regarding Build Times, I was hoping someone could share any and all tips regarding how to optimize my build times. I am currently sitting at 8 minutes for my project. That means I can possibly reach a max of 37 updates in a month.
Can anyone help me figure out why netlify tries to upload 500+ files on every single tiny change.
In case this could be useful for others, so far what I have found out today regarding optimizing build times:
Stopping Auto Publishing
One of the things that Netlify does, auto-publishes on every tiny commit to the “main” usually master branch. To be honest, this is a really cool thing but is very costly. This becomes a pretty big issue as every other day, Dependabot puts in like 5-6 pull requests. Merging one triggers the first deploy and then merging the rest trigger a second deploy after that first one is complete. Sometimes if there are conflicts that take quite a while for Dependabot to resolve it can even cause a necessary third/fourth build needed.
Another thing that I can imagine will cause build times to start skyrocketing is automated deploy previews. Once again, although this is an incredibly amazing feature it will cause the build times to increase way too much.
I tried using Gatsby-Plugin-Netlify-Cache recently but with its default settings I saw only a 1 minute decrease in the build times. I am going to try enabling the options to cache the public folder even though there is a warning attached to it. Hopefully this should fix some of the build times on my site. If anyone has had luck with this plugin so far please let me know.
If there are any other tips I would really appreciate it.
Hey there! these are really good questions and awesome places to start. We are currently working on a guide with tips in order to keep build times as lean as possible. We will be sharing it soon, and we’ll update the thread here when its live.
In the mean time, if anyone else has ideas, please chime in!
note that stopping auto publish doesnt affect build times at all. it just means that your latest build does not automatically get “published” aka go live. if you pull in 5 dependabot PR’s in short succession, Netlify actually builds the first and last PR’s and skips the middle builds, by default. this happens regardless if you have auto publish on or off.
i think one thing worth exploring is prebuilding the images you need for gatsby-image. gatsby-transformer-sharp is pretty unnecessarily reprocessing the images every single time. you may wish to switch to gatsby-transformer-cloudinary or similar solution to upload files one time to a cloud gatsby-transformer-cloudinary - npm. alternatively check optimized images into git.
@perry Would really love an guide with tips to improving my build times. I know very little about this area as I always just said ehh whatever itll build sooner or later. This prevented me from improving my code and project from being better so this change to is much appreciated as it encourages me to think differently and try to improve my project as much as possible.
Thank you @fool . That was a really good read. I think Swyx might be right for my specific case that the issues are being caused from the issue in gatsby for having every file being uploaded every small commit due to the hash change.
@swyx The reason I turned off the auto-publish was that every small change would trigger another 8 minutes of build time in my project. The total build time would increase too much. Dependabot ate up 16 minutes minimum every couple days for me. I’ve hit over 3000 minutes in build time last month from just 1 project. I hope to try and decrease this a lot this month.
EDIT: It was actually a lot more than 16 minutes. I just realized I also had deploy previews enabled for all branches so every PR the bot made it would also trigger another build meaning it probably hit around 60 minutes every few days.
In order to keep the project relatively up to date, I am thinking of creating a small script hosted on my VPS that will just trigger a build once a day if the github repo had been updated that day. This should keep my project(and future ones) mostly up to date.
Thank you so much for those links. Those are incredibly helpful.
most definitely, @Skillz. I think we will have it edited and vetted and ready to put in front of the world very soon. It’s a good point that you are highlighting, this does encourage people to optimize slow code, which is better for a variety of reasons. Glad you are thinking of it in this way.
Update on this: I enabled the cachePublic option and the build times have reduced from 10 minutes to now being 4 minutes. This is going to help dramatically with my build times as thats about a 60% decrease. Note: There was a warning attached to using the cachePublic option so I will update here if i run into any issues. So far it is working as expected.
There is no way for you to trigger a build with the cache cleared unless the prior build on that branch/PR failed (or it’s your production branch), but we can do it for you if you ask.
Just a quick update, there is one small adjustment we could make for Gatsby sites is to disable telemetry. You can read more about it here.
In general this could be a bad idea to disable on your local machine and i’m not saying you should or shouldn’t do by any means.
But i could understand if someone runs a bunch of builds and it needs to squeeze out every second then this could have some good effects.
You can disable this in the Netlify App → Site Settings → Build & Deploy → Environment
