Why is the size of my cached depencies much bigger than my repo?

I have a simple single page static website without frameworks, no build command and using netlify automatic deploys, but for some time I have noticed that the cached dependencies are over 300mb and taking most of the build time.
There are no package.json or any other files related to NPM, so why is the size so much bigger than my repo branch (25mb)?

image

1 Like

It might also contain the commit history of the repo. It’s just a guess.

And is there something I can do about it? I thought it would only fetch resources from the current branch.

I’m not sure. But, do you really need to? It hardly takes a few seconds and since it’s a simple static page, I suppose you’re not otherwise spending your build minutes. Even if it takes 1 second to build and deploy your website, you’d be charged for 1 full minute, I suppose.

It’s more about knowing why this is happening in a small scale project and if there is something that can be done about it, since for me working in bigger projects means that I need this type of details clearer in case it happens later on.

What do you mean by that? Even if a build time is bellow 1 minute, the time registered/charged is a full minute?
Nothing in this page explain it clearly : https://www.netlify.com/pricing/faq/

Yes that’s my assumption as that’s how it works everywhere, for example calling rates or internet charges. If we pay for a minimum unit, anything below it is also charged till the closest greatest value. Since Netlify doesn’t show my build minutes usage in seconds, I think it considers only in minutes.

If that’s really the case, maybe their page should be updated to clear this information out, guessing how you will be billed based on how other services work its not an assumption that a customer should be making.

For example Github actions has this info in their page, even though it’s kinda hidden and could be more prominent:

The quote they are using:

If your account’s usage surpasses these limits and you have set a spending limit above $0, you will pay $0.25 USD per GB of storage per month and per-minute usage depending on the operating system used by the GitHub-hosted runner. GitHub rounds the minutes each job uses up to the nearest minute.

Yes, I agree with you. If what I’ve said above is true, there should be a warning. I think a member of support team will soon confirm this and pass on the change if required.

The same is true for your original question, the docs mention only about dependencies as the cache, which is not true in your case (or even mine actually). I don’t use any Node Modules either, however, my website’s size is around 787 MB (including the .git folder):

image

And Netlify downloads a cache of 864 MB:

image

So yeah, I think the .git folder size does matter as Netlify ‘clones’ your repo and while performing a git clone, the .git folder is also transferred.

Thank you, I’ll be paying attention to their official answer. Anyway, this was a good amount of information :D.

Yep, looking at my project the .git folder is responsible for all the ‘extra’ cache size seen in my screenshot.
Since this is the case, why is it happening? Intended behavior or overlook in their part? I don’t think the .git folder is required for the functioning of the website.
I can see some bigger projects having huge caches in this situation.

As I mentioned in my previous reply, Netlify ‘clones’ your repo with the git clone command. By the nature of git, the commit history (.git folder) is downloaded with this command. That’s how git works, and it’s beyond Netlify’s control. You can try to reduce the size by removing your commit history, however I’m not sure if this will work. A folder that’s just initialized as a git repo would also consume double the amount of space. So, a 100 MB website will have 200 MB cache size due to this folder.

Yes, they can face such a problem and that’s where the extra build minutes help.

Yeah, I got that, but then the ‘cache’ is just the cloned repo, plain and simple? Then I suppose netlify just uses the git history to control which files were changed and don’t require other files to be updated.
I’m not trying to sound obnoxious, just there are some details of how things work in the background that were missing in my understanding of the platform.
Thank you for the patience :).

While it might be true in your case as your website doesn’t need to be built as such, others have websites made in Static Site Generators, and they need all the files of the repo anyways. So, the repo is cloned, the build command is run, the generated files are then compared with the files of the previous deploy and updated files are uploaded to the live website. This is the general process.

Got it.
About the cloning process, the way it is now, the whole repo being cloned is the only option or can you specify a single branch? I mean, in the control panel you can select a branch, but as far I got from that it doesn’t affect the fact that the role repo is being cloned.

Yes, git clone clones the entire repo by default. Another git thing sadly.

Since fetching a single specific branch on git is possible through the --single-branch while cloning any repo, would be nice to see something like that introduced as an control option while importing a repo to netlify.
Thanks, this helps a lot!

Hi, @Douglas_William. You might also examine the cache itself to see what is contains.

There is a build plugin here to assist with debugging the build cache:

You can install it here if you are interested:

http://app.netlify.com/plugins/netlify-plugin-debug-cache

It should help you to confirm exactly what the cache does and does not contain. It will take a lot of the guess work out of this troubleshooting.

If there are other questions, please let us know.

Hi @luke, thank you for the suggestion, I will give a look.
Could you please clarify the information bellow?

not 100% sure this is the perfect answer to your question, but have you seen this post already?

Hi, @Douglas_William. Internally, the granularity of the “build minutes” calculations is by the second.

We bill by the minute. We calculate by the second. You can see more details of your team’s build minutes use here:

You can see there on March 2, there were six builds in the day. However, the total build minutes use for the day is only one minute.

For example, if you have two builds each which are 15 seconds long (for a total of 30 seconds), your total usage would be 1 minute (because the total is rounded up to the nearest minute).

Another example, two builds, one which is 2 minutes 30 seconds and one which is 2 minutes and 31 seconds.

This would be 5 minutes and 1 second total so that monthly usage (assuming only these two builds in the billing cycle) would be 6 minutes as the 5:01 is rounded up to the nearest minute.

So, iIf you have 60 builds that are all 2 seconds each, that is going to bill as 2 minutes and not 60 minutes.

1 Like