Issues deploying with Yarn and private GSAP registry

cassieevans · October 28, 2021, 4:49pm

Hello there Netlify pals!

Cassie here from GreenSock.

The issue

Our users are having some issues deploying sites on Netlify that are using yarn and our private GSAP registry - which requires a token set as an environment variable through your UI.

There is no issue deploying when users are using npm. Only Yarn.

The site

I’ve managed to recreate the issue on this site:

site name: loving-wilson-ff4891

I have pushed up a yarn.lock file as per this article

if you deploy a site that includes a yarn.lock file in the base of the repository, we’ll use Yarn to install dependencies.

And followed the ENV var/Private module instructions given in this doc

The site works locally. But it won’t deploy. Is there something going on with the Netlify build container?

It would be wonderful to get this resolved.

I hope I’ve given you all that you need.

Look forward to hearing from you.

Cassie

relevant log below

5:46:08 PM: > Successfully installed Yarn 1.22.10! Please open another terminal where the `yarn` command will now be available.
5:46:08 PM: No yarn workspaces detected
5:46:08 PM: Started restoring cached node modules
5:46:08 PM: Finished restoring cached node modules
5:46:09 PM: Installing NPM modules using Yarn version 1.22.10
5:46:09 PM: yarn install v1.22.10
5:46:09 PM: warning package.json: No license field
5:46:09 PM: warning gsap-trial-bonus-plugins@1.0.0: No license field
5:46:09 PM: [1/4] Resolving packages...
5:46:10 PM: [2/4] Fetching packages...
5:46:13 PM: error An unexpected error occurred: "https://npm.greensock.com/@gsap%2fbusiness/-/business-3.8.0.tgz: Request failed \"403 Forbidden\"".
5:46:13 PM: info If you think this is a bug, please open a bug report with the information provided in "/opt/build/repo/yarn-error.log".
5:46:13 PM: info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
5:46:24 PM: Error during Yarn install
5:46:24 PM: Build was terminated: Build script returned non-zero exit code: 1
5:46:24 PM: Creating deploy upload records
5:46:24 PM: Failing build: Failed to build site

hrishikesh · October 29, 2021, 11:02am

Hi @cassieevans,

Does the same issue occur with npm?

Additionally, what happens if you store the credentials in .yarnrc instead of .npmc when using Yarn?

cassieevans · November 1, 2021, 2:11pm

Hi Hrishikesh,

No issues with npm at all.

We are recommending people use npm instead - but this doesn’t really seem like a solution

With a .yarnrc the build also fails.

hrishikesh · November 3, 2021, 11:38am

I’m going through this:

github.com/yarnpkg/yarn

Private registry: Fetching packages not possible

opened 10:25AM - 10 Oct 17 UTC

closed 03:19PM - 20 Mar 19 UTC

ptusch

cat-bug help wanted high-priority triaged cat-backend needs-repro-script needs-discussion

**Versions** * `yarn@1.2.0` * `node@8.6.0` * `Linux (arch) x86_64` **Do yo…u want to request a *feature* or report a *bug*?** `Bug` - or - `missing feature` **What is the current behavior?** `yarn add PRIVATE_PACKAGE` does not work - I'm never authenticated (this happens with a freshly set-up `~/.yarnrc` with not pre existing `~/.npmrc`) **If the current behavior is a bug, please provide the steps to reproduce.** * Get a private npm repository (I used artifactory) * Make full access bound to a specific user (note: read only needs user as well - no anonymous!) * Setup the registry in yarn: `yarn config set registry https://my_cool_artifactory/api/npm/cool_npm_repository` * Publish a sample package (this works nicely) * Attempt to get that package using `yarn add PRIVATE_PACKAGE` **What is the expected behavior?** I suppose I should get the package. After I set up the repository and a 4xx error is returned, I expect a CLI query asking for my username, email and password (drop the username and email when those are already entered in `~/.yarnrc`) **More descriptive stuff** Okay, we have an artifactory server which serves as both, a proxy and a private repository container. There is no anonymous access to the server so even read access is bound to a proper log in. I started from scratch with no pre existing .npmrc. I created my `~/.yarnrc` by doing this: `yarn config set registry https://my_artifactory/and/its/repo` `yarn config set always-auth true` `yarn config set strict-ssl true` `yarn config set cafile /path/to/root/cert` After this set up I invoked `yarn login` and entered my username and email. Publishing a package is no problem since I'm asked for my password here. The workflow for the publishing seems to be * Query for version * Query for password * create a token with HTTP PUT to artifactory * use? this token to publish my package * Delete the token from artifactory with HTTP DELETE After this, I'm left as I started - without a token since it was deleted (and it was probably in the RAM anyway). When I then invoke `yarn add` to add my private package, I'm not queried for my password which results is a 403 (Forbidden). Yarn doesn't evaluate this issue and merely reports it back to me. I didn't find a way to set a proper auth token. I played a bit with postman and was able to create a valid token for myself but not sure how I'd use it after this (maybe there is a hidden CLI flag?). This seems to be the wrong way anyway. Manually grabbing a token to make the package manager work sounds counter intuitive to me. Looking into other threads didn't help either. Most "solutions" seem to be work arounds. The only solution that *seemed* to work was to put my plain credentials into the `~/.npmrc` and work with that. But that only worked for a direct private package with no private packages as dependencies there. I suppose the credentials are stripped from the URL after the first iteration This also happened another day while testing with a second `~/.npmrc` I guess I'll try to add the token I created with postman in the `~/.npmrc` manner and add it like //MY_REGISTRY:TOKEN Anyway, I hope this is any helpful and maybe someone can help me out. Cheers Edit: Adding the token to the `~/.yarnrc` Didn't help either.

but by giving it a short read, I’m not sure if the bug still exists, or was fixed or what’s the status exactly? Some say it works, some say it doesn’t.

Would you be able to go through that thread?

cassieevans · November 4, 2021, 2:28pm

I’m taking a look - I’ve also taken this to the yarn discord. Lets see!

cassieevans · November 4, 2021, 2:57pm

Just adding again though that this works on my machine - just not on Netlify.

So I’m not really sure what more I can do from my end.

hrishikesh · November 5, 2021, 10:55am

Have you tried this on a Ubuntu machine? Maybe it’s just happening on Linux-based devices?

Since npm works fine, I believe it would have to do something with Yarn’s implementation, and because it works locally, it could be happening only on a few systems.

cassieevans · November 8, 2021, 12:59pm

I haven’t tried on an Ubuntu machine and I don’t have one easily available I’m afraid.

I really need to open an issue with the yarn team so they can fix it. Build stuff isn’t really my area of expertise so I’d appreciate any information about the issue and the setup of the Netlify build container.

I can’t replicate this issue outside of Netlify so it’s pretty difficult for me to play middle-man here.

If this was just a personal issue I would use npm - but I don’t think I can just tell our users ‘you can’t use yarn with netlify’ - that seems less than ideal.

cassieevans · November 10, 2021, 2:37pm

Bumping again - could I please have any relevant information from your side to take to the yarn team. Thanks.

5:46:13 PM: info If you think this is a bug, please open a bug report with the information provided in "/opt/build/repo/yarn-error.log"

hrishikesh · November 10, 2021, 2:48pm

You might want to use a virtual machine or a Docker image.

These files are deleted after the build finishes, so those logs are not available. If you need them, you might have to use the build script to store the file to an external server before the build exits.

I could try testing it on my end, would it be possible for you to share the repo?

cassieevans · November 10, 2021, 2:53pm

I’m very very front-of-the-front-end mate - sorry. Build stuff isn’t my area at all.

I just need to be able to take something to yarn to try and get this sorted because we keep getting netlify users in our forums asking us what the issue is

If you can test it on your end that would be wonderful.

hrishikesh · November 10, 2021, 3:04pm

I tried this repo: https://github.com/cassieevans/test-yarn-netlify, seems to be private?

cassieevans · November 10, 2021, 4:59pm

Oh sorry hrishikesh - I only just realised you meant git repo and not netlfiy site.

I’ve made it public - that is the right one.

hrishikesh · November 10, 2021, 6:03pm

Yeah, but sadly that also has the private module you’re using and I’d need the token.

Is there a read-only token or something of that sort that I could get to try this locally? You could share one via PM if needed.

hrishikesh · November 11, 2021, 12:45pm

Thanks for the token @cassieevans, I tried this on my Macbook for now, but I got the Not Found error too.

These are the logs:

Arguments: 
  /Users/hrishikesh/.nvm/versions/node/v17.0.1/bin/node /Users/hrishikesh/Desktop/test-yarn-netlify/node_modules/.bin/yarn

PATH: 
  /Users/hrishikesh/Desktop/test-yarn-netlify/node_modules/.bin:/Users/hrishikesh/Desktop/node_modules/.bin:/Users/hrishikesh/node_modules/.bin:/Users/node_modules/.bin:/node_modules/.bin:/Users/hrishikesh/.nvm/versions/node/v17.0.1/lib/node_modules/npm/node_modules/@npmcli/run-script/lib/node-gyp-bin:/Users/hrishikesh/Desktop/test-yarn-netlify/node_modules/.bin:/Users/hrishikesh/.asdf/shims:/usr/local/Cellar/asdf/0.8.1_1/libexec/bin:/Users/hrishikesh/Netlify/google-cloud-sdk/bin:/Users/hrishikesh/.nvm/versions/node/v17.0.1/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion.app/Contents/Public:/Users/hrishikesh/Desktop/test-yarn-netlify/node_modules/.bin

Yarn version: 
  1.22.17

Node version: 
  17.0.1

Platform: 
  darwin x64

Trace: 
  Error: https://registry.yarnpkg.com/@gsap%2fbusiness: Not found
      at Request.params.callback [as _callback] (/Users/hrishikesh/Desktop/test-yarn-netlify/node_modules/yarn/lib/cli.js:67029:18)
      at Request.self.callback (/Users/hrishikesh/Desktop/test-yarn-netlify/node_modules/yarn/lib/cli.js:140883:22)
      at Request.emit (node:events:390:28)
      at Request.<anonymous> (/Users/hrishikesh/Desktop/test-yarn-netlify/node_modules/yarn/lib/cli.js:141855:10)
      at Request.emit (node:events:390:28)
      at IncomingMessage.<anonymous> (/Users/hrishikesh/Desktop/test-yarn-netlify/node_modules/yarn/lib/cli.js:141777:12)
      at Object.onceWrapper (node:events:509:28)
      at IncomingMessage.emit (node:events:402:35)
      at endReadableNT (node:internal/streams/readable:1340:12)
      at processTicksAndRejections (node:internal/process/task_queues:83:21)

npm manifest: 
  {
    "name": "gsap-trial-bonus-plugins",
    "version": "1.0.0",
    "description": "Try all GSAP bonus plugins for free on CodeSandbox.\n\nSimply add gsap-trial to your dependencies or fork this template.",
    "main": "index.html",
    "scripts": {
      "preinstall": "yarn config list",
      "start": "parcel index.html --open",
      "build": "parcel build index.html"
    },
    "dependencies": {
      "gsap": "npm:@gsap/business",
      "parcel-bundler": "^1.6.1"
    },
    "devDependencies": {
      "@babel/core": "7.2.0"
    },
    "resolutions": {
      "@babel/preset-env": "7.13.8"
    }
  }

yarn manifest: 
  No manifest

Lockfile: 
  No lockfile

Not sure if it helps you to get to the bottom of this issue.

cassieevans · November 11, 2021, 1:20pm

We’re using different versions of node so maybe that’s a step in the right direction.

Thanks so much for your help pal. I’ll take this over to the yarn team now.

Hugely appreciated. Have a lovely day!

hrishikesh · November 11, 2021, 1:26pm

If that makes a difference, you could try setting the same Node Version on Netlify by using the NODE_VERSION environment variable.

_brettsmith · December 15, 2021, 3:59pm

Hi @cassieevans were you able to resolve this issue? I’m currently running into the same exact thing!

cassieevans · December 16, 2021, 9:48pm

No - I’m still without answers I’m afraid.

Here’s the thread on the GSAP site.

Are you having issues with our private module or another one?

cassieevans · December 16, 2021, 9:49pm