We use npm (not yarn) and we want to make sure our netlify builds install the versions of packages in our package-lock.json file.
Your docs here say that you run npm install to install dependencies at build time, but npm install will overwrite the package-lock.json file. I read in another forum that you evaluated using npm ci (which is the npm command to do what we want: install the packages listed in the package-lock file) but decided against that.
So, what method can we use to ensure that a particular SHA in our repo will always install the same dependencies? (I guess we could have a build command check to see if the package.lock file is dirty and throw an error, but this seems REALLY roundabout and will cause unnecessary failures). I figure we can’t be the only dev shop who wants predictable build behaviors, I just haven’t figured out the netlify way to do this…
But it’s not the same dependencies that ran in our CI system or on the developer’s machine! this is super basic stuff… I mean REALLY basic… are you saying you don’t support ANY form of dependency locking!!!
Another problem is that we will have different netlify apps for our different tiers (production, staging, etc) and so the locking for one app won’t work for the other one.
And yes, I know about having different branches on the same netlify app but we have a complicated DNS setup, need to proxy websockets, etc. We can’t use your dns and we can’t manage this with one app.
This is really bad-- we’ll have to do our builds elsewhere and then push pre-built files to netlify… what a disappointment!
Perhaps I am not fully understanding the complexity of you dev/build/prod setup. Is locking the dependency versions in the package.json using semver as per package-json | npm Docs not an option?
Yes, that’s an option but that’s HUGE pain in the rear for us when we are updating or changing dependencies. We WANT to be able to take on patch updates, etc. all the time, we just want that to be a dev-time decision and to know that once we’ve got a SHA that CI results and deployments in different environments are all going to be using the same code.
This command is similar to npm install, except it’s meant to be used in automated environments such as test platforms, continuous integration, and deployment – or any situation where you want to make sure you’re doing a clean install of your dependencies.
Note that this is why services like CircleCi use npm ci..
Also, please note that your service is not the only place that we build… we need reliable builds… Your caching is a way to speed up builds, not a way to guarantee reliable builds. You shouldn’t recommend relying on the cache to guarantee predictable builds.
One other thought here: you mentioned that you don’t use Yarn, but not that you can’t use Yarn. Yarn does have a way to handle this: yarn install --frozen-lockfile Would that be possible to try?
Eventually we will move to yarn. I’ve used yarn in the past, but just started with a new company that isn’t using it yet.
Note that this PR calls out all the reasons that this change makes sense-- I understand that you don’t yet support npm ci but I think the right answer from support/the team is “Yes, we know that there’s a feature gap/flaw in our build system but we don’t have a fix for it yet” (as opposed to suggesting that folks rely on caching to ensure idempotency).
Hey @uooq,
That’s fair! Thanks for the feedback. I do want to note that it’s not an official Netlify recommendation to consider caching a dependency-locking strategy. In digging around internally, I found that our build engineering team is having (and has been having for many months) a conversation about npm ci vs npm install. It’s pretty crunchy and would be a significant, breaking change to how our build image and caching work. I’ve cross-linked this thread there to voice support for the change, and so the team can get more context on customer use cases where npm ci would be the best solution. We’ll be sure to let you know if there are any updates on that conversation.
OK, we’re having a parallel discussion in our company on whether to switch to yarn or run our builds in a separate CI pipeline and upload to netlify or whether to just use our own aws s3+cloudfront setup.
I feel like you could at least OFFER npm ci as a different option (i.e. have three node build types: npm install, yarn & npm ci and then not break existing caching behaviors).
I mean, it’s JUST a different form of the npm install command… if it was a flag that went to npm install then everything would work the same way as before, right? It can’t be THAT big of a breaking change (and if the dev team is saying so, they either don’t understand or there’s something SO wacky about your system that I can’t even imagine it).
Like if there was a FLAG you could pass to npm install this would work, right?
I wish it was as easy as adding a flag! But it actually is that big of a breaking change.
npm ci relies on ~/.npm, not a node_modules directory. In fact, if node_modules is present when npm ci runs, the directory will be removed. So we would have to switch to caching ~/.npm, and we would need to evaluate the impact on the time it takes to download/upload cached files that way instead.
There’s also the fact that if there are any discrepancies between package.json and package-lock.json (i.e. missing dependencies), the build will exit with an error. What if a build takes minutes to detect this discrepancy before failing? Our customers pay for build minutes, even for failed builds, so we need to make sure we do our due diligence here and minimize the amount of time before failure as much as possible.
Plus npm ci requires a package-lock.json, which not everyone knows how to generate or include. And even if we design our npm ci implementation to fall back to npm install when npm ci breaks, which is something being discussed, each execution path has to be written, tested, and incrementally rolled out behind feature flags to make sure we don’t break previous functionality that customer’s are relying on.
Within that, this discussion is just one of many discussions about our build system that is happening between our product and build engineering teams. As I mentioned earlier, I’ve linked this thread to voice support for npm ci and we will be sure to follow up here with any updates. In the meantime yarn install --frozen-lockfile will be the best alternative.
To clarify, I wasn’t suggesting that you convert ALL npm projects to npm ci, but that you add it as a 3rd option: 1) npm install, 2) yarn 3) npm ci.
I would love to move to yarn but it will take my team a while to get there. They have a very complicated internal build/release pipeline and I’m working hard to get it out of a lot of unmaintainable in-house scripts into services like Netlify. But this is a big blocker-- If I had the option of paying for more build minutes and not caching dependencies, I would. The point of the matter is that there’s NO WAY that I can predict what will be in my node_modules directory right now.
NO web app should use npm install for build pipelines. Period. It’s a terrible practice. Yes, it’s a deficiency of npm that they don’t have a way to checksum the packagelock or something like that to know if it’s necessary to nuke the packages, but it seems like other providers have figured out a way.
Do you let us take manual control of our package installations and look at our own caches like like CircleCI?
Netlify is far from averse when it comes to modifying/fixing their systems as evidenced by the CDP issues outlined here and here and the subsequent fix here.
As @jenoutlined such as change as you are mentioning
and also mentioned previously
Does your current situation not sound like that of Netlify changing to or offering npm ci as an alternative?
I understand, and I appreciate that. I’m just pointing out that there’s a quicker win to be had by offering a third alternative.
On my end I’m a HUGE fan of Netlify and have been touting this as a game changer at my new company. I haven’t used npm in years and years (I’ve been on yarn for a long time) and I’m trying to push a lot of changes through at a company with a lot of devops tech debt and a slightly cautious attitude to risk. So, please understand that despite my pushing this, I remain 100% Team Netlify and part of my tone has been more due to surprise than anything-- it seemed so unlikely that you didn’t have any way of enforcing consistent dependencies with npm.
Thanks for pushing for new things-- I wish I could wave a wand and get us onto yarn but it means tackling a lot of self-hosted, managed-by-hand CI environments. We’ll figure something out.
Quick question: Suppose if during our netlify builds we check to see if the package-lock.json file has been modified and throw an error if it has. Will the dependencies still be cached?
Apologies for the very late response here- I missed your last reply. And many thanks for the kind words and push for improvements on our end.
To your question: if the build errors and fails, dependencies will not be cached. What I’m not sure about, though, is how you would re-overwrite the package-lock to what it was originally, and build with that instead of the overwritten package-lock.
Yep, I get it… just kicking the tires on some short-term options-- in that case we’d know we needed to test stuff and perhaps could update our committed package-lock.json and try again… eventually I think all roads lead to yarn… there’s just a chicken-and-egg situation: I don’t want to update our internal build/deploy process to yarn, I want to deprecate it, but we need to switch to Netlify before we can deprecate. (Sharing all this just as flavor so you all get a sense for customer use cases…)
I have just spent hours of trying to understand why the site on netlify behaved differently
–
I was very surprised that packae-lock.json is not respected, and that there is no option to use npm ci. I excepted full predictability when it came to what was being deployed. For me this feels so fundamental, that I might have to look into alternative services
Thanks so much for taking the time to add your opinion here. I hear your expectations, and have added your thoughts to our open issue that one of our teams is aware of. We will follow up on this thread should we have updates.
