[Support Guide] Using private NPM modules on Netlify

Maybe something like this in package.json?

"preinstall": "if [ ! -z \"$NETLIFY\" ]; then echo \"//npm.pkg.github.com/:_authToken=${NPM_TOKEN}\" >~/.npmrc; fi || true",

Thanks for sharing that @loof and welcome to the Netlify community site! :slight_smile:

@fool perhaps now is a good time to add built in support for at least private packages on GitHub? It looks like a lot of folks will be moving their private packages that way soon npm is joining GitHub | The GitHub Blog

This currently doesn’t seem to be possible with build plugins (beta). Related issue over here: Add onPreInstall event · Issue #1149 · netlify/build · GitHub

This is still an absolute chore with netlify. With any other system I can just authorize npm with a token, or have a special build-script, etc.

Why is the solution for private npm on netlify a workaround? It’s an exceptionally common use-case.

My ultimate suggestion is that you just have support for a build-netlify task, which, if it exists, is run by netlify instead of the regular build task. This would make this process a lot easier.

Seems pretty straightforward to me - set an environment variable and use it. Could you explain in more detail what’s so tedious about this 1 line config?

1 Like

Hi team, we’re in the same situation as many of you in that we’re not comfortable editing our .npmrc file locally. The solution that we’ve got working is to edit it as part of the “Build command”
Steps:

  1. Add NPM_TOKEN to environment variables in netlify
  2. Disable netlify’s install - we achieved this by adding the NPM_FLAGS = --dry-run environment variable
  3. Edit .npmrc - Update your build command to:
    echo '//npm.pkg.github.com/:_authToken=${NPM_TOKEN}' >> .npmrc && npm install && npm run build

Hopefully this helps!

1 Like

None of the solutions given are really acceptable.

Directly adding the .npmrc file causes local npm installs to fail due to missing the variable ie: //npm.pkg.github.com/:_authToken=${NPM_TOKEN}. Given that developers normally login to npm using the npm login command locally, there should be absolutely no reason for them to also need to set NPM_TOKEN on their machine just so that npm install can work on their machine.

Using all these workarounds with dry-run means that we get billed more than we would have without the dry run. I think that speaks for itself as to why it’s not a viable long term solution.

And we shouldn’t have to be maintaining a custom preinstall hook in our package.json for this. This is really something that should be handled by the netlify.toml file, such as a “preinstall_command” hook.

After diving into this issue a bit, I found this. Adding NPM_TOKEN as an environment automatically adds the registry information for NPM. Is it possible to have this functionality added for github as well, such as GITHUB_TOKEN?

Ideally there should be a pre-install hook for build commands.

2 Likes

Hey @logo,

I’m sorry to hear that none of these solutions really cut it for you. I can see that this has been filed in our build-image repo already: Add authentication string to .npmrc for Github packages · Issue #423 · netlify/build-image · GitHub.

Because there are workaround and alternates, it usually means that a request like this falls down the pecking order. Though, if/when it happens, we’ll keep you in the loop!

I’d also like to thank you for your efforts and sleuthing! I’ve gone ahead and refunded the build minutes you’ve used this period so at least you don’t need to worry about that.

@Scott that was me that opened that :slight_smile:

I was unfortunately unable to get the workaround involving --dry-run to work, which while inefficient seemed like the best way to get everything working without any custom scripts. The dry run was still attempting to contact the registry and check the packages on the first (uncached) request, which would fail the build.

We’ve instead opted for manually setting the token as a local environment variable on our developers machines for the time being.

Thanks for the update though, and the refund. I understand this is a fairly new and niche problem right now, so I’d also like to voice my support for adding custom pre-install commands to the build pipeline. I imagine that is something that’s a lot more useful to a lot more people anyways, and would help to easily fix this as well!

Hey @logo,

Makes sense! I appreciate your understanding and, although it’s not the working method we’d like, I’m glad that you have a solution which is enough to keep you going :+1:. We’ll be sure to feed back both in the linked issue and here if there are any developments.

1 Like

I am having similar issues with the pre-subscribed solutions as @logo. Netlify should expose functionality that can be commonly found in CI/CD pipelines (like CircleCI) for building assets from private repos. Not all people use NPM or Github for hosting private assets, my company uses artifactory. Having a local .npmrc file breaks local development for us, and it has been challenging to find a solution that allows us to pull assets from our private repository.

Here is a sample snippet of code from a config.yml file that we use to set these variables during a CI/CD build.

commands:
    configure_npm:
        description: "Configure the npm client to use private artifactory as registry"
        steps:
          - run: npm config set _auth $NPM_AUTH
          - run: npm config set email $NPM_EMAIL
          - run: npm config set always-auth true
          - run: npm config set registry https://website.com/artifactory/api/npm/reponame/

Couldn’t the netlify.toml file expose similar type steps to make the build phase easier to configure?

Good news, a fix for this issue has been put on the Netlify roadmap :smiley:
Refer to these GitHub issues:

This is genius and works like a charm!

Wanted to say that @mpan-wework’s suggestion worked for me, and it felt like the least amount of compromise. The tricky bit is that the developers on my team need to use both private npm packages and packages published to a self-hosted GitLab registry, so there are multiple config steps required. I was able to create an .npmrc file in an arbitrary folder within my project(s), which is setup to use environment variables that I’ve set at a team level. I then added a line to netlify.toml to set the NPM_CONFIG_USERCONFIG option.

Inside netlify.toml:

[build.environment]
  NPM_CONFIG_USERCONFIG = "./netlify/.npmrc"

And then I have a file within my project at ./netlify/.npmrc:

# Auth for private GitLab registry modules
@my-scope:registry=https://${gitlab_instance_url}/api/v4/packages/npm/
//${gitlab_instance_url}/api/v4/packages/npm/:_authToken=${GL_REGISTRY_TOKEN}

# Auth for private npm modules
//registry.npmjs.org/:_authToken=${NPM_TOKEN}

The variables NPM_TOKEN and GL_REGISTRY_TOKEN are both exposed in the Team / Sites / Global site settings.

The downside: every project we build needs to include this file and this configuration in netlify.toml, but this is fairly minor.

The upside: everything “just works” without forcing developers to alter their local development environments to accommodate builds in Netlify. This file is only used in the context of Netlify, and it only needs to be updated if and when we add more registries.

7 Likes

Thank you so much for taking the time to write this up in such detail, @grahaml!

1 Like

@grahaml thanks for sharing such a beautiful solution!

This is indeed the best solution, as it allows developers to have their own git-ignored .npmrc in the main folder with the tokens set directly in the file (so no export or set env variable script has to be called) and to leverage the env variables in Netlify properly. Thank you so much!

Does NOT work with YARN throws error like it completely ignores npmrc.

Could not find package “@scope/package” in the NPM registry
info Visit yarn install | Yarn for documentation about this command.
Error during Yarn install

It seems like it doesn’t understand .npmrc file set in the netlify.toml

How do you set authentication in yarnrc instead?

"@scope:registry" "https://npm.pkg.github.com"

Hey @FlatFlowerFrogFlag, not my area of expertise though there appears to be some workarounds over in the yarn repo :slight_smile: