Build cache not working. Size limits?

frangio · November 16, 2019, 3:37am

Hi! I’m trying to optimize a complex build by storing things in /opt/build/cache, but things don’t seem to be in the cache in a second build. I’m still in the process of debugging, but I wanted to first ask: is there a limit to the size of what will be persisted in the build cache? This site can result in about 2GB of stuff in there.

Thank you.

fool · November 17, 2019, 7:17pm

There’s no size limit on what you store; 2Gb should work fine.

frangio · November 17, 2019, 7:35pm

Thank you!

Is there anything you know of that could result in failure to persist that directory across builds?

fool · November 18, 2019, 7:12pm

Nothing systemic, as long as they are on the same branch - we only intend to use cache on same branch or PR as it started on. So, you do of course need to be resilient to it not being present, e.g. on a new PR.

You could try listing the directory before your build completes, and at restore time (before your build starts) to ensure that it does/doesn’t save and restore what you are expecting, and you can see what we try to do in the code, here:

github.com

netlify/build-image/blob/xenial/run-build-functions.sh#L32-L53


      
          
          
# CI signal
          export NETLIFY=true
          
          
YELLOW="\033[0;33m"
          NC="\033[0m" # No Color
          
          
# language versions
          mkdir -p $NETLIFY_CACHE_DIR/node_version
          mkdir -p $NETLIFY_CACHE_DIR/ruby_version
          mkdir -p $NETLIFY_CACHE_DIR/swift_version
          
          
# pwd caches
          NETLIFY_JS_WORKSPACES_CACHE_DIR="$NETLIFY_CACHE_DIR/js-workspaces"
          
          
mkdir -p $NETLIFY_JS_WORKSPACES_CACHE_DIR
          mkdir -p $NETLIFY_CACHE_DIR/node_modules
          mkdir -p $NETLIFY_CACHE_DIR/.bundle
          mkdir -p $NETLIFY_CACHE_DIR/bower_components
          mkdir -p $NETLIFY_CACHE_DIR/.venv

This file has been truncated. show original

frangio · November 19, 2019, 11:36pm

I’m still unable to get this to work.

This is an example deployment: Netlify App

Look for the lines that say /opt/build/cache contents. I’m trying to cache the /opt/build/cache/xdg directory. It doesn’t exist at the beginning, and it’s been created by the time it finishes. But a following build doesn’t seem to pick it up again.

fool · November 20, 2019, 12:48am

I don’t see that we’d archive that path since it’s not one of the ones we create…are you using one of those paths mentioned in my selected lines from the github script?

You might also consider other implementations such as cache-me-outside - npm which are known to work, in inspiring your design.

frangio · November 25, 2019, 1:42pm

It turned out that I had accidentally made the cache contents way larger than I thought. It may have been like 6GB at some point. This happened because my program was putting stuff in ~/.cache, a.k.a. $XDG_CACHE_HOME, and Netlify was caching it without my knowledge as the “pip cache”. I think it would be better to either cache only the pip directory or improve the logs to make it clear that it is potentially more than just the pip cache.

What’s weird is that my efforts of clearing the cache and rebuilding didn’t seem to have any effect. It felt like the cache was corrupt in some other way.

I started a new Netlify site (with the same sources) with a fresh cache and was able to get it all working.

Thanks for the help @fool! I hope this will help someone else as well.