Images not cached

Support
, ,
1

Site name: https://glistening-eclair-de7f24.netlify.app

There is an issue with this site build and images not being cached.

When the build completes, I see this error: Failed during stage 'building site': Command did not finish within the time limit … but the site files are still successfully uploaded.

Also the ‘images’ directory is empty in the following file: https://develop--glistening-eclair-de7f24.netlify.app/cache-output.json

When I test the Netlify build locally, images are cached.

I believe this could be happening because the build isn’t finishing in a specified time limit? Is there a way we can increase the timeout limit for this site to see if the image folder is eventually fully cached?

Or any other advice would be appreciated.

I can provide the full build log if need be, it was too long to add to this post.

Many thanks.

2

I’ve upgraded the build time limit of that site to 45 mins. Try another build and let us know the results.

1 Like
3

Thanks very much @hrishikesh! The build completes without any errors now, and I think the images are being cached. The first ‘clear cache’ build took around 25 mins, since the build time limit increase.

Should I be worried that it said Cache file count: 3753? This is just the .cache directory (3753 files).

The dist/images folder contains all the images required (8101 files) but they’re not reported in cache stats.

However when I ran the build a second time, since the time limit increase, it took 1 minute 20 seconds! Which is amazing.

I’m just confused as to why the images aren’t recognised in the cache stats or in cache-output.json

Hopefully won’t be an issue.

Thanks again mate.

4

Based on the documentation for the cache.list: @netlify/cache-utils - npm, it goes only to depth 1 and that’s being used to determine the number of files: netlify-plugin-debug-cache/index.js at master · netlify-labs/netlify-plugin-debug-cache · GitHub

You can try changing the depth, or ignore this as long as it’s working fine for you.

5

Ahh that makes sense, thanks @hrishikesh… One last question; is there a way we can specify the age of cached items?

It seems every now and then, the deploy process loses track of the images it needs and reverts to re-fetching everything, taking around 17 mins to build. Then the next few builds immediately after that are back to taking 1-3mins…

Or do you have any idea on what kind of code changes would cause all files to be re-downloaded? Realise this could also be an 11ty specific problem.

6

Hi @jrh I would have to see a deploy URL or deploy ID for your site where you have seen a 17 minute build time to be certain, but this does sound like a config setting where the images are marked for refresh which causes them to be re-downloaded.

7

Hey @Kai-Mavyn, thanks for your reply. This deploy ID took 19 mins 24 secs: 636e2df5cc7328000928b46a

The two deploys after that took 4m 2s and 1m 25s respectively.

9

Not sure why that build takes longer, to be honest. I can see, Eleventy take up 15 min of that build time, but why, that I do not know. It doesn’t seem to mention doing anything about the images in that build log (but does mention a lot about .html and .njk files). Why do you think it has something to do with the images not being cached? Maybe the next 2 builds were faster because they could use the data from .cache folder?

10

So during the Eleventy build it fetches images before writing any template files (.html and .njk). I believe that is where the delay is caused.

I can only compare it to what I test locally, both running 11ty or Netlify builds. The only time the build takes nearly this long locally is when it is done cold and has no template or image cache to reference…

To add: if i run netlify build locally with Eleventy in debug mode and no cache, it appears to be the lack of image cache that causes a similar lengthy build time.

11

While this isn’t enough to cause even 5 minutes of delays, when I look between your slower and faster deploys, I see this pattern:

Slower deploys more assets (which means more assets have to be checksummed, and uploaded):

image

Faster deploys do less:

image

This article explains why it is faster to change fewer files in each deploy:

Like I said I doubt this is everything, but it may be a contributing factor.

12

Yep it’s a fair point.

But for context a build this morning took 4 mins and generated the following: 541 new files uploaded. 508 generated pages and 33 assets changed.

Then a few moments later, same branch took 20 mins and generated: 510 new files uploaded. 508 generated pages and 2 assets changed.

So I feel confused by that haha.

13

Just to expand on this further, it seems like the extended build times are isolated to when the build is triggered by a different team member. Would there be any reason this could be the case?

I recently did 5 builds in a row and each took around 2-3 minutes, my colleague then merged a branch into develop and the build took 30 minutes.

Looking back on it, this is likely what has caused the problem in the past. The code base and and content are identical, so it’s really confusing.

Cheers,
J

16

That would be a bit of a surprise, since your build cache is shared between all team members. However, I do see that a long build:

reported that the 11ty cache wasn’t used at all:

vs one that didn’t suffer the delay used the 11ty cache:

. So, question for you: what makes that build different than one that uses the cache? This is not something Netlify knows, because netlify did not make Eleventy nor configure your build pipeline :slight_smile: