Cache takes very long

Hi!

I have a repository (vue-instantsearch.netlify.app) which for some reason has a recursive folder which makes the cache take several minutes every time on the “pip” step (even though I don’t use pip). Is there a way to maybe disable the cache altogether, or specifically for pip? This is causing builds to time-out even if it’s not needed :slight_smile:

build log

online available here: Netlify App


1:43:49 PM: Build ready to start

// ... snipped most of the build out, it's not what takes the longest

1:54:47 PM: Done in 427.68s.

1:54:47 PM: Skipping functions preparation step: no functions directory set

1:54:47 PM: Caching artifacts

1:54:47 PM: Started saving node modules

1:54:47 PM: Finished saving node modules

1:54:47 PM: Started saving yarn cache

1:54:47 PM: Finished saving yarn cache

1:54:47 PM: Started saving pip cache

2:00:04 PM: Execution timed out after 15m0s

2:00:04 PM: Error running command: Command did not finish within the time limit

2:00:04 PM: Failing build: Failed to build site

2:00:05 PM: failed during stage 'building site': Command did not finish within the time limit

2:00:11 PM: Finished processing build request in 16m19.240633879s

When I “clear cache and deploy” it usually does finish within the time limit for the record.

Thanks!

hey there,

i wonder if this thread is useful for you?

1 Like

When I run the script, there’s no .cache folders, except in one space, which is unfortunately recursive. My script already runs rm -rf on that directory first though, so I don’t think it’s exactly that. Can you see what’s in a build’s cache or disable folders from being cached?

hey @Haroenv, we just wanted to let you know we haven’t forgotten about you and are asking a few questions internally about this. we’ll get back to you when we can. thank you for your patience.

1 Like

No worries, I understand that issues like these can take a longer time.

One thing that I definitely noticed, which might make the issue harder, is that I’m doing yarn install in child directories, with a dependency to file:../../. Normally Yarn is supposed to honour the files key in that root package.json, but it doesn’t seem to do so and copies the whole folders recursively (including other node_modules). I noticed that each of those node modules (nested too) includes a .cache folder with a few items in it.

However, I have a script which should remove the node_modules before it gets cached: vue-instantsearch/clean-node-modules.sh at master · algolia/vue-instantsearch · GitHub

PS: also a happy birthday to you :smiley:

1 Like

How do you do your yarn installs? The yarn i we do for you has some pretty specific caching settings, and if you use yarn yourself without them you could end up with an inconsistent or “interesting” cache. The settings we use are all in this long shell script, but I know you’re a yarn whiz and bet you can pick out what’s relevant:

It is possible you’re missing some of our cache directories too which are all in there.

I wonder why you use rsync like that, instead of rm -rf’ing? That might be more reliable?

Just wanted to let you know that:

  • our dev team looked into this for us
  • they found some things in how we handle the cache, that can cause slowness like you are experiencing
  • have filed a couple of enhancement requests to improve those behaviors
  • and are already working on the one that should have the biggest impact.

We’ll follow up here when that last one is shipped!

1 Like

Thanks Chris, really appreciate how you handle support :slight_smile: we’ve interacted a few times and you’ve always been stellar!

Thanks for submitting this issue!

We’re rolling out a change to the way we handle the pip cache. I turned it on for your account. Could you let us know if you see better performance when restoring the cache?

(If you’re curious, the change is here: Don't cache full .cache dir by vbrown608 · Pull Request #443 · netlify/build-image · GitHub)

Hi there,

I’m Eunjae, a colleague of Haroen. I re-ran a failed job and it’s still taking too long when saving pip cache.

In the end of the log above, you can see

10:41:09 AM: Started saving pip cache
10:52:28 AM: Execution timed out after 15m0s

Let me know if there’s anything I can try :slight_smile:

Hi Eunjae,

I took another look and the site you linked is using our Trusty build image, which is no longer getting bugfixes. We strongly recommend that customers select the Xenial image for builds. You can read more about the differences here: [Support Guide] [Deprecated] What are the differences between the Trusty and Xenial build images?

Would upgrading to the Xenial image be possible for your team?

2 Likes

this fixed it, thanks a lot Vivian :slight_smile:

1 Like

Yay, so glad to hear it worked out!