Netlify mono-repo, Yarn, and caching

Support
,
1

The site is: gifted-antelope-58b104.netlify.app

Here are the build logs for an example failing build.

I have a file-tree that looks like:

.
├── netlify.toml
├── package.json
├── packages
│   ├── discovery
│   │   ├── netlify.toml
│   │   ├── package.json
│   ├── frontend
│   │   ├── package.json
│   ├── hardhat
│   │   ├── package.json
│   ├── lib
│   │   ├── package.json
│   ├── pixel-art-react
│   │   ├── package.json
│   ├── social
│   │   ├── package.json
│   └── ui-components
│       ├── package.json
└── yarn.lock

I am deploying the discovery package, using packages/discovery as a base.

Notably, in packages/discovery/package.json, we have a dependencies entry

"@supercool/ui-components": "*"

And in packages/discovery/ui-components/package.json we have another internal dependency:

"@supercool/lib": "*",

Everything works fine when I deploy with a cleared cached, but when I deploy with a cache, it cannot find the linked workspace dependencies.

  1. Working build: deployed with a cleared cache
  2. Failing build: deployed with a populated cache

The working build was deployed with a cleared cache. Then right after I pushed a non-consequential file change and the next build failed. If I clear the cache and retry, the build succeeds.

From the website, the build settings are:

  • Base directory: packages/discovery
  • Build command: yarn validate-env && yarn build
  • Publish directory: packages/discovery/.next

And my packages/discovery/netlify.toml:

[build]
# postinstall package.json scripts don't run when the dependency is cached, so
# we need to manually call those commands when we build in Netlify.
command = "yarn validate-env && yarn build"
publish = "./.next"

[build.environment]
NETLIFY_USE_YARN = "true"

[[plugins]]
package = "@netlify/plugin-nextjs"

[context.beta-discovery]
[context.beta-discovery.environment]
NEXT_PUBLIC_APP_URL_SCHEME = "https"
NEXT_PUBLIC_APP_HOST = "discovery-beta.netlify.app"
NEXT_PUBLIC_APP_DOMAIN = "discovery-beta.netlify.app"

NEXT_PUBLIC_SUPABASE_URL = "https://gnqzipfybyrapoqfiqkj.supabase.co"
SUPABASE_URL = "https://gnqzipfybyrapoqfiqkj.supabase.co"
NEXT_PUBLIC_SUPABASE_ANON_PUBLIC_KEY = <redacted>

RECOMMENDER_MODEL_LOAD_FROM = "supabase"
RECOMMENDER_MODEL_STORAGE_PATH = "2022-05-30_similarity_rankings_15k.txt"

NEXT_PUBLIC_POSTHOG_API_KEY = <redacted>

# Define these in web UI
#
# SUPABASE_SERVICE_ROLE_KEY
# DATABASE_URL
# SIMPLEHASH_API_KEY

# Also, serverless functions that need to access variables at runtime (as
# opposed to build-time) need their environment variables defined in the UI.
# These are:
#
# RECOMMENDER_MODEL_LOAD_FROM
# RECOMMENDER_MODEL_STORAGE_PATH

Thanks for the help!

3

Hey @dbmikus,

This seems to be an issue with how we’re caching Node Modules for Workspaces. Currently, the Node Modules folder in your cache turns up to be empty. However, the next build believes that the cache has the files and thus, proceeds with that. Thus, it eventually fails.

This is already being discussed with our engineers as a few other users are seeing it too. We’ll let you know once we’ve more info.

Meanwhile, you’d have to continue deploying by clearing cache.

4

Thanks, sounds good. Looking forward to when this gets resolved :grin:

5

Hey @dbmikus,

If you’re still having this issue, could you try to set an environment variable with the name NPM_VERSION and value as 8.4.0 (or lower)?

If that doesn’t change anything, could you please share your package.json?