I tried multiple different things but my understanding is that I need to set the base directory to e.g. repository-root/site-1.
I found this anwser here about including private Github packages via npm preinstall script and installing dependencies seems to work. I do get an error message though but the build continues to run (sometimes):
8:16:17 AM: Installing NPM modules using Yarn version 1.22.10
8:16:17 AM: npm ERR! code ENOWORKSPACES
8:16:17 AM: npm ERR! This command does not support workspaces.
Eventually the build fails when trying to resolve one of those private packages:
8:23:55 AM: failed Building production JavaScript and CSS bundles - 47.038s
8:23:55 AM: error Generating JavaScript bundles failed
8:23:55 AM: Can't resolve 'component-library-1' in '/opt/build/repo/site-1/src/components'
Because the site doesn’t have a yarn.lock file (only in the root), I specified the NETLIFY_USE_YARN flag in the netlify.toml file.
Apologies for the delay on this one! We’ve taken a look and a couple of things stood out to us:
I’m seeing signs of mixed npm and yarn usage. Something you can do immediately: set npm or yarn version that you are using locally. Can you set a specific version and then run locally?
I don’t think you need private packages as described in the other post you linked - it looks to us like you have the files in the repo. In this instance we suggest you follow our yarn or npm docs for monorepos to install from your local files instead.
Could you try making those changes and let us know if that helps? If you’re still seeing an issue with the build after making those changes, can you share what error remains when you attempt to build?
I think the npm signs were leftovers of some wild testing to make it work. I specified the YARN_VERSION via env variable to match my local version but it didn’t change anything.
I need to use (private) packages for versioning. I don’t want site-2 to be affected when I’m working on a shared package and site-1.
So the error remains the same: Installing the dependencies (including the private Github packages) seems to work but the build command fails (GatsbyJS) with the message:
8:23:55 AM: failed Building production JavaScript and CSS bundles - 47.038s
8:23:55 AM: error Generating JavaScript bundles failed
8:23:55 AM: Can't resolve 'component-library-1' in '/opt/build/repo/site-1/src/components'
Thanks for the reply, but I don’t think this thread offers the solution for me.
In my case, I’m using a monorepo but I don’t really want Netlify to threat it as such. That means I don’t want site-1 to resolve to local components but to the Github registry. So in my understanding I need to set the base directory to point to site-1 instead of the root and then create the .npmrc file via preinstall command mentioned in this answer and installing the (private) packages seems to work according to the logs but as soon as the code runs the build command those packages can’t get resolved.
I’m not sure how this is related but the logs also show yarn workspaces detected even though this shouldn’t be the case if I specify the base directory, correct?
This is the problem that has led Amelia, Hillary, and myself to wonder what exactly your build is tryign to do:
12:26:44 PM: Found yarn version (1.22.10) that doesn't match expected (1.22.5)
npm ERR! Workspaces not supported for global packages
12:26:44 PM: npm ERR! A complete log of this run can be found in:
12:26:44 PM: npm ERR! /opt/buildhome/.npm/_logs/2022-04-11T19_26_44_018Z-debug-0.log
We are still trying to run npm for you, and failing, and you’ll want to address it. Perhaps you don’t need package.json which is what our build script keys off of:
…but instead only yarn.lock?
You can see that right after that, we switch to yarn, but use what must be a wrong version, since I am pretty sure that yarn v1 does not support workspaces at all:
… which the logs do helpfully point out to you just after that:
12:26:46 PM: npm ERR! code ENOWORKSPACES
12:26:46 PM: npm ERR! This command does not support workspaces.
So, in light of all of that, plus the fact that you seem to have conflicting code in the monorepo (local copies of the code you want to fetch as a package), we’d want you to create better reproduction steps for us that do a minimal thing:
have no wrong config files (e.g. package.json)
have no local copies of any packages you want to install remotely
use only your intended version of yarn
and attempt to fetch packages during our dependency installation - which right now is not happening at all due to the above errors (instead, you are managing your own dependencies, here: Netlify App - and that is not something we intend to debug; we want to debug our build system being used in the intended way which is us managing dependencies).
So, can you create that simpelr reproduction for us?
Thanks in advance for your help in troubleshooting!
I ran a bunch of tests and it turns out that deleting the package.json file in the root and replacing the script for generating the .npmrc file with an actual .npmrc file fixed the build issue.
However, the package.json file in the root is required for a yarn/lerna workspace (and therefore not wrong). I could add it to .gitignore and add an additional setup step to the readme but this feels wrong. I’m wondering why Netlify looks at the package.json file in the root even if I defined a different base directory? Is there a way to explicitly tell Netlify to treat the repo not as a monorepo (e.g. via env variable)?
I’m wondering why Netlify looks at the package.json file in the root even if I defined a different base directory? Is there a way to explicitly tell Netlify to treat the repo not as a monorepo (e.g. via env variable)?
Typically, non-monorepos don’t have a base dir set, but there is no explicit “treat this not as a monorepo” flag today. As far as I can tell, the point of the build script that detects package.json does it for all repo types though, so we’d want a one-size-fits-all solution. From the codeblock I mentioned in my post, I wonder if you have any suggestions for how to do it differently in your case? Yours is the main occurrence of this being a problem across our millions of customers, so I start to wonder if your use is atypical instead of ours, but as I am not a JS dev, I wonder if you have any thoughts that I could present to our dev team about the right way to handle your use case and others
The problem was, that the local package versions matched the versions referenced in the site-1 package.json file and Netlify attempted to use the local instances rather than the ones in the Github registry but the build/dist folders aren’t committed to Git so there was nothing to resolve and the builds failed.
To fix this, I modified my build script to build the local packages first cd ../component-library-1 && yarn build && cd ../site-1 && gatsby build. And the .npmrc generation script mentioned earlier must be triggered within the package.json file of each site rather than in the root directory.
However, that raises two questions:
Why do the logs indicate that a monorepo is not supported even though it is?
11:20:59 AM: Installing NPM modules using Yarn version 1.22.10
11:20:59 AM: npm ERR! code ENOWORKSPACES
11:20:59 AM: npm ERR! This command does not support workspaces.
Is the base directory setting confusing?
It might be my own personal misconception of the setting, but I assumed that the base directory would define the entry point of the build process and that all files/folders outside will be ignored. That is clearly not the case. The definition in the Netlify docs is misleading in my opinion but I might be wrong.
The base directory setting prompts our buildbots to change to the specified directory to detect dependencies and perform caching during the build process. It’s useful for building from a monorepo or subdirectory.
Hey @Robin_Zimmer , great to hear you figured out the issue!
The npm ERR! code ENOWORKSPACES error that you’ve referenced is pointing to an issue with yarn or npm version. Perhaps an updated yarn or npm version or npm package is needed?
The base directory is where our buildbots look for package.json/.nvmrc/etc before starting a build. If it’s not set, it defaults to the root directory. You mention this is not the case. Could you provide an example for us to take a closer look?
Hope this helps but let me know if I’ve misunderstood your questions!
