Cache Jekyll gems added during preinstall script

Support
,
1

Hi there! I’ve been struggling with this problem for a few months & finally have found a solution, but I think there might be a better way to handle it to better take advantage of Netlify’s power. Any help is greatly appreciated!

Briefly summarize the issues you have been experiencing.
For several of my sites, the build & deploy would regularly fail unless I clicked the “Retry deploy & clear cache” option. These are Jekyll sites, but use Gulp as a task runner to compile CSS, JS, and run bundle exec jekyll build. The build command I have Netlify use is npm run build (as opposed to jekyll build). Any time the cache was used, the build would fail because it couldn’t find a particular gem public_suffix (a dependency of a dependency of Jekyll). This is because bundle install was run as a preinstall script, and for whatever reason, Netlify wasn’t caching the result of that script and the preinstall wasn’t run once a cache is created.

NPM Scripts used:

"scripts": {
    "preinstall": "bundle install --path vendor/bundle",
    "start": "gulp",
    "build": "gulp build --jekyllEnv='production'",
}

I do have a Gemfile that includes the Jekyll gem as well.

What have you tried as far as troubleshooting goes? Do you have an idea what is causing the problem?
I’ve been able to fix the problem of those gems not being available by adding a prebuild step that duplicates the preinstall, like this:

"prebuild": "bundle install --path vendor/bundle",

The “issue” I’m running into with this solution is that now Netlify isn’t caching any of the Jekyll gems used, which slows down the deploy. It isn’t a significant amount, but I’m sure there is probably a better way to do this that would allow Netlify to cache those gems so they don’t have to be reinstalled every time a build is done.

Live Site URL: https://healthsecurity.csis.org/
GitHub Repo: GitHub - CSIS-iLab/health-commission

Thanks in advance for any help!

2

Hi @jschrag !

You can check out how we handle the caching in our build scripts which are open source: build-image/run-build-functions.sh at xenial · netlify/build-image · GitHub . If you install your dependencies elsewhere, we won’t see them, so best to make sure whatever process you use installs your gems and npm modules using the cache directories specified in that script. I don’t know the intimate differences between prebuild and preinstall for npm, so not sure why that makes a difference, but making sure they end up in the correct directory should allow them to be available upon rebuild :slight_smile:

PS: You have a pretty big problem in your build command - it’s two commands separated by a semicolon which means we could easily publish a broken site if your npm run build fails but the other command succeeds. Recommended to chain with && which won’t allow that to happen, instead of ; !

PPS: Great question - you gave enough details for anyone to help you and were very thorough in the details. Thank you so much!!

1 Like
3

Thanks so much for the feedback and for the help! I’ve taken another look at my build logs and it looks like Netlify is automatically detecting & installing gems based on my Gemfile. Removing both the preinstall & prebuild script has seemingly done the trick :crossed_fingers:. So I think there was something happening where the gems were being installed multiple times (and maybe in different places) that was causing the build script to fail.

Thanks again for the help! I’ve also updated the build command to use &&. Thanks for pointing it out - I had seen it done both ways, but didn’t know the difference between them. :slight_smile:

1 Like