I’m having trouble with the new monorepo deployment options to the point that I can’t deploy a site that uses a standard monorepo setup with yarn workspace.
Netlify have the option to change the base directory for sites now which means I can built from multiple directories - which is great - but it only uses yarn if it finds a yarn.lock in the directory that sites are built from. In most monorepos I have multiple sites but also packages that are shared between sites. With a yarn workspaces setup you would not have a yarn.lock for each site or the package directory:
/yarn.lock
/site-a/ /* no yarn lock so will use npm */
/site-b/ /* no yarn lock so will use npm */
/packages/shared-module/ /* no yarn lock */
Since we don’t have lock files in package directories it will not install correctly because yarn workspaces is aware that /packages/shared-module/ is a local package but npm is not.
Sorry that is quite technical and very specific to monorepos but it really makes the new Netlify features difficult to use with real monorepos and it could possibly be “easily” fixed. Or perhaps you know a workaround?
I’m trying to write about this at the moment and I’d love to promote these features.
Hi @MadeByMike thanks for taking the time and opening the post. Our monorepo feature is built for repos that have their frontend in a subdirectory along with the yarn.lock. One of my peers has suggested some alternatives to this use case at Build monorepo project on netlify - #4 by marcus. We understand that the experience is not entirely optimal. We do look forward to releasing some more documentation around this support in the coming weeks. Hope that helps. Thanks!
Hmmm, this rules out many of the main use-cases for monorepos. I might even say that although you support multi-package repositories you can’t really support monorepos without being able to build from yarn workspaces or similar.
Maybe I can’t argue too strongly since definitions and people’s understanding of “monorepo” varies widely. But I can say it’s not going to match a lot of peoples expectations.
How hard would it be to detect a yarn workspace set-up (or allow a user to signal this) and change how the build works?
Any chance could get me an indication if this is going to be on the roadmap?
@MadeByMike Totally hear you on this. It is definitely top of our minds and something we are considering into the roadmap. I am unfortunately unable to give out a timeline yet. But, we definitely would love to have a seamless experience here.
Hey! Any update on this? I’m having the same exact issue. Will have 100s of static sites in a monorepo in the coming months and would love to get them all on Netlify without much hassle.
As I mentioned in the other thread, no change, but we’d be happy to try to help you come up with a workaround if you give us more details about what’s failing. Try to keep it in a single thread so we can help you most efficiently though - thanks in advance!
Hi Chris, all the details about what is failing are in the issue already. Unfortunately without a change there is no work-around.
One small change that might allow sites with monorepos to install correctly is to allow users to specify that they want to use yarn for installations without requiring a yarn.lock file.
A better solution would be to look into how yarn workspaces work, detect when a site is part of a monorepo and install packages accordingly.
I appreciate the intention behind the offer to come up with a workaround, but I think don’t think you will have any luck there without a deeper dive into monorepo installations. This is a really good intro guide on how they are typically set-up: https://monorepo.guide/getting-started
When that issue was written, we didn’t have even the partial monorepo support that we have now. I think it would be worth working through a demo setup again to see if the new support changes the situation at all - I suspect we could potentially work around things in ANY site that has a one-subdirectory-per-build setup. Perhaps a workflow like:
set base directory to that subdirectory
make sure you set $YARN_VERSION to the version you want
if that subdirectory has no yarn.lock file, we won’t run yarn, but you can run it as part of your build command
if that subdirectory has a yarn.lock file and you DON’T want to run yarn there, you can set $YARN_FLAGS to something like --dry-run to make that a no-op.
if you need yarn run from the parent directory, a build command like cd .. && yarn i && cd projectdir && yarn build may do the trick.
Some time ago I experimented with Yarn Workspaces and Netlify’s recently released better support for monorepos that @fool mentioned (netlify.toml per site in subdirectory). I discovered a setup that might solve some of the above mentioned problems.
Assumptions:
You use Yarn Workspaces with more than one site (or one site in subdirectory),
You set your base dir in Netlify UI to e.g. packages/website to use per-site netlify.toml file
You want Netlify to install dependencies using yarn (without additional commands, changing directory or installing first with NPM and then with Yarn)
You want Netlify to cache dependencies between builds
So, here’s the fun part. Based on facts that:
You can run yarn install in a Workspace directory instead of project root
Netlify requires yarn.lock to exists in a base directory to use Yarn instead of NPM
And Netlify will pick up your netlify.toml and install dependencies with Yarn
Unfortunately, this does not eliminate the problem with proper caching of node_modules/. Fortunately, Yarn uses local cache directory that Netlify persists between builds. That speeds up things a bit. But for the issue to be fully resolved we have to wait for a native solution in build image.
I might be able to do some testing to see if I can get it to install although I think even if I trigger yarn it’s not going to install dependencies from the base directory.
For example if I am deploying website here in the directory structure below, can I access the base directory to install dependencies including shared local packages, e.e. the lib folder?
Chris, I’m well aware of the Support for Monorepos Netlify released on October 9, and spoke with several of your team about it before creating this issue on October 19.
It worked during my testing. Yarn installed all dependencies for all workspace packages, as one would expect.
It does so, because when yarn install command is run in of the workspace subdirectories, it uses top-level yarn.lock instead of package.json. It’s a yarn documented feature, not a workaround The only tricky part is empty yarn.lock inside subdirectory. It will trigger Netlify to use yarn and we rely on the fact that yarn is smart enough to not use it and look higher in hierarchy. It’s not a documented feature though.
This is super helpful. Honestly Netlify should document the current confusing DX, and work to improve the setup for Monorepos if they’re going to advertise their support on their blog. Yarn Workspace is most common setup, so ideally would be pretty optimized with that setup and Lerna as well.
Hey @bsgreenb,
Thanks for this feedback! I’ve added it to an internal issue we have open about improving support for Workspaces and Lerna. One improvement since dmgawel wrote in March is that the “save an empty yarn.lock” part of that workaround can now be replaced with setting the newish NETLIFY_USE_YARN env var to true.
Please, please update this on Yarn support on Netlify which is the first page to come up when searching google for ‘netlify yarn’. NETLIFY_USE_YARN is such a better solution than wonky file detection that breaks when using workspaces.
This support forum is very useful but such good information needs to be better surfaced than this.
The only tricky part is empty yarn.lock inside subdirectory. It will trigger Netlify to use yarn and we rely on the fact that yarn is smart enough to not use it and look higher in hierarchy. It’s not a documented feature though.