We’re working through some issues where some of our JS contributors are committing a package.json update for our deployed app without also updating package-lock.json. This presents problems when trying to create reproducible builds with deterministic dependencies. Most recently some of our builds have failed on Netlify but work fine locally (while silently overwriting package-lock.json and creating a dirty git tree). I want broken builds to break in CI, and automated builds should absolutely never overwrite the lockfile.
I’ve noticed that JS builds without a yarn.lock file automatically run npm install before turning over to the build command in my netlify.toml file. The yarn builds seem to run yarn install. Neither of these are correct. Both tools have ways to pull deps solely from their respective lockfiles and verify that the package.json is compatible with the locks. If the lockfiles don’t match package.json, the build will fail. For npm the correct invocation is npm ci and for yarn it appears to be yarn install --frozen-lockfile.
Is there any way to run npm ci instead of npm install, or otherwise have Netlify skip the install step so I can run npm ci myself without the automatic install step screwing up my checkout?
Our build system isn’t super flexible, but that pattern works for most folks and to change it would break builds on thousands of sites. Fortunately, you can work around it in a few ways:
don’t have a package.json, package-lock.json or yarn.lock in the root of your repo (or in your base directory if you have one set). Then we won’t automatically run anything.
you could instead “fake us out” and use something like $YARN_FLAGS set to --dry-run to make the auto-install a no-op, or maybe you want the --package-lock-only option to npm to create the lock, which you can then use? Not sure exactly what will best solve your use case. There’s a similar $NPM_FLAGS. You can set those in our UI, or in the netlify.toml config file.
Then you can run npm ci or yarn ci or whatever build command you like, in whatever way you like. Note that if you want to use anything other than npm - you’ll need to first INSTALL it- we have a VERY bare build environment by default, so you’ll need npm i yarn && yarn ci if you use yarn.
Note that unless you do some manual dependency caching at that point, you will have also opted out of that, since we run those installation commands with a special cache directory as shown here:
…so you might want to do some explicit cache management using a package like cache-me-outside - npm . That isn’t officially supported, but works well for many folks.
Sorry I don’t have a more direct way to accomplish your goal today. It’s interesting that this is the first we’ve heard of this need (not of the desire not to auto-run yarn/npm, but to use npm ci instead of npm install. Perhaps what you really need is a
Hello there, even I am looking for a similar solution. Currently Netlify fails to install packages from private registries (not private packages), and there doesn’t seem to be any way around it. Tried asking questions too, but no satisfactory answer was given. But, npm ci would have solved the problem
This is again related to private packages distributed via registry.npmjs.org. I was talking about private registries - not packages alone. For example, I have a private registry at r.privjs.com and I need to install a few packages hosted at r.privjs.com - but netlify always seems to request the package from registry.npmjs.org even though we clearly mentioned the registry URL in package-lock.json file
Sorry to be dense - I don’t use any private registries, so the distinction is lost on me. I’m suggesting that you use the example I mentioned above, and substitute //r.privjs.com for //registry.npmjs.org . Will that not work for your use case?
Um, that does not work because netlify (for some strange reason) always tries to install the packages from registry.npmjs.org even though private registry is specified in the package-lock.json file
I’d love to bump this topic and see if Netlify can reconsider an option to enable respecting a package-lock.json file by using npm ci instead of npm install.
I understand that this could be a breaking change, so potentially making this an optional upgrade that becomes mandatory, like NodeJS runtime updates, could be considered. There are other great comments in that GitHub issue that is now archived, otherwise I’d post in that issue.
Since this thread is pretty old, can you confirm a bit more about your use case, @p-ob ? My understanding (which is not expert!) of the current situation is that:
if you have a package.json with exact versions we should always follow it, regardless of package-lock.json
whether you do or don’t have a package-lock.json, npm ci should never update dependencies vs what’s in the build cache (?), vs npm install will always try to install the latest spec that matches what is in package.json (e.g. version “5” will go to latest x, and latest y within x, for 5.x.y).
I am curious though - if you have a package.json with the exact right versions specified, I would think npm install does the right thing, regardless of any other situation. Is that not what you experience in our CI? If not, could you elaborate a bit more on what happens? If so, could you explain a bit more about your use case that requires older versions that you aren’t willing to “lock” in package.json directly, which should remove any difference in behavior?
In my situation (and likely many others), my application has a number of transitive dependencies of the packages I directly rely on. As such, explicitly going through and pinning each would be a difficult task, especially when npm has produced a solution via the lockfile and the npm ci command.
I definitely can pin the versions in my package.json, but if a dependency downstream isn’t pinned, the next npm install is going to get updated transitive dependencies.
The npm ci command ensures build-time integrity by only installing dependencies as they are in the lock file.
I hope that explains my use case for being able to respect the lock file!
I will ask our dev team for their advice on this. I am certain I can coach you in working around our automatic npm install so you can run whatever npm commands you want, but that would also avoid our dependency caching (or worst case, make it seem as though they were cached and cause things to fail unexpectedly), so I am hoping to avoid that (which I consider to be “the nuclear option”) in favor of something easier to manage / that still leverages our build cache.