Separating Client and Serverless Functions Base Directory In Mongorepo Project

gatsby not found error would appear because there might not be gatbsy in your package.json or your node_modules folder might not exist in direct hierarchy.

1 Like

So that error only shows up when I use the subshell approach to the command not when I set the base folder to client.

Like this build command doesn’t work

'(cd client && npm run build)

Gatsby is in the package.json and node modules as it does successfully builds in my previous setup.

Edit: I’ve edited my post so that it’s a bit clearer

Alright, your edit does help. So I think (not sure about this), Netlify doesn’t support having node_modules in the parent folder like it does on our local system, thus the error.

Maybe an official support team member would confirm this.

1 Like

Oh okay, so when I try to run the command in the subdirectory. It looks for the node modules in the base directory. Because the node_modules are where the build command is but not where the command is being “executed”

Is that what’s causing the error?

Yes, that’s what I think is happening.

1 Like

Do you know of a way for me to reach my end goal of being able to separate the UI and the serverless functions?

I can’t think of a way you could do that without Netlify supporting node_modules in a different folder than the base path, but I think it does. I still feel that cd client && npm run build should have worked. Could you make a test repo for us to try?

1 Like

I’d be happy to make a test repo. Here it is GitHub - LuisOsta/netlify-test.
And here’s the output of the Netlify build:

8:25:50 AM: Build ready to start
8:25:51 AM: build-image version: d84c79427e8f83c1ba17bcdd7b3fe38059376b68
8:25:51 AM: build-image tag: v3.6.1
8:25:51 AM: buildbot version: df42d44cc6a0381cf657a671baf049c03981bafe
8:25:52 AM: Fetching cached dependencies
8:25:52 AM: Failed to fetch cache, continuing with build
8:25:52 AM: Starting to prepare the repo for build
8:25:52 AM: No cached dependencies found. Cloning fresh repo
8:25:52 AM: git clone GitHub - LuisOsta/netlify-test
8:25:53 AM: Preparing Git Reference refs/heads/master
8:25:54 AM: Different publish path detected, going to use the one specified in the Netlify configuration file: ‘client/public’ versus ‘/’ in the Netlify UI
8:25:54 AM: Different functions path detected, going to use the one specified in the Netlify configuration file: ‘serverless’ versus ‘’ in the Netlify UI
8:25:54 AM: Different build command detected, going to use the one specified in the Netlify configuration file: ‘(cd client && npm run build)’ versus ‘’ in the Netlify UI
8:25:54 AM: Starting build script
8:25:54 AM: Installing dependencies
8:25:54 AM: Python version set to 2.7
8:25:55 AM: v12.18.0 is already installed.
8:25:56 AM: Now using node v12.18.0 (npm v6.14.4)
8:25:56 AM: Started restoring cached build plugins
8:25:56 AM: Finished restoring cached build plugins
8:25:56 AM: Attempting ruby version 2.7.1, read from environment
8:25:57 AM: Using ruby version 2.7.1
8:25:58 AM: Using PHP version 5.6
8:25:58 AM: Started restoring cached go cache
8:25:58 AM: Finished restoring cached go cache
8:25:58 AM: go version go1.14.4 linux/amd64
8:25:58 AM: go version go1.14.4 linux/amd64
8:25:58 AM: Installing missing commands
8:25:58 AM: Verify run directory
8:25:59 AM: ​
8:25:59 AM: ────────────────────────────────────────────────────────────────
8:25:59 AM: Netlify Build
8:25:59 AM: ────────────────────────────────────────────────────────────────
8:25:59 AM: ​
8:25:59 AM: ❯ Version
8:25:59 AM: @netlify/build 9.1.3
8:25:59 AM: ​
8:25:59 AM: ❯ Flags
8:25:59 AM: deployId: 6033beeecc0a302fa44ae1eb
8:25:59 AM: mode: buildbot
8:25:59 AM: ​
8:25:59 AM: ❯ Current directory
8:25:59 AM: /opt/build/repo
8:25:59 AM: ​
8:25:59 AM: ❯ Config file
8:25:59 AM: /opt/build/repo/netlify.toml
8:25:59 AM: ​
8:25:59 AM: ❯ Context
8:25:59 AM: production
8:25:59 AM: ​
8:25:59 AM: ❯ Installing plugins
8:25:59 AM: - @netlify/plugin-lighthouse@1.4.2
8:25:59 AM: - netlify-plugin-gatsby-cache@0.3.0
8:25:59 AM: - netlify-plugin-inline-critical-css@1.2.0
8:27:28 AM: ​
8:27:28 AM: ❯ Loading plugins
8:27:28 AM: - @netlify/plugin-lighthouse@1.4.2 from netlify.toml
8:27:28 AM: - netlify-plugin-gatsby-cache@0.3.0 from netlify.toml
8:27:28 AM: - netlify-plugin-inline-critical-css@1.2.0 from netlify.toml
8:27:29 AM: ​
8:27:29 AM: ────────────────────────────────────────────────────────────────
8:27:29 AM: 1. onPreBuild command from netlify-plugin-gatsby-cache
8:27:29 AM: ────────────────────────────────────────────────────────────────
8:27:29 AM: ​
8:27:30 AM: No Gatsby cache found. Building fresh.
8:27:30 AM: ​
8:27:30 AM: (netlify-plugin-gatsby-cache onPreBuild completed in 228ms)
8:27:30 AM: ​
8:27:30 AM: ────────────────────────────────────────────────────────────────
8:27:30 AM: 2. build.command from netlify.toml
8:27:30 AM: ────────────────────────────────────────────────────────────────
8:27:30 AM: ​
8:27:30 AM: $ (cd client && npm run build)
8:27:30 AM: > client@0.1.0 build /opt/build/repo/client
8:27:30 AM: > react-scripts build
8:27:30 AM: sh: 1: react-scripts: not found
8:27:30 AM: npm ERR! code ELIFECYCLE
8:27:30 AM: npm ERR! syscall spawn
8:27:30 AM: npm ERR! file sh
8:27:30 AM: npm ERR! errno ENOENT
8:27:30 AM: npm ERR! client@0.1.0 build: react-scripts build
8:27:30 AM: npm ERR! spawn ENOENT
8:27:30 AM: npm ERR!
8:27:30 AM: npm ERR! Failed at the client@0.1.0 build script.
8:27:30 AM: npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
8:27:30 AM: npm WARN Local package.json exists, but node_modules missing, did you mean to install?
8:27:30 AM: npm ERR! A complete log of this run can be found in:
8:27:30 AM: npm ERR! /opt/buildhome/.npm/_logs/2021-02-22T14_27_30_512Z-debug.log
8:27:30 AM: ​
8:27:30 AM: ────────────────────────────────────────────────────────────────
8:27:30 AM: “build.command” failed
8:27:30 AM: ────────────────────────────────────────────────────────────────
8:27:30 AM: ​
8:27:30 AM: Error message
8:27:30 AM: Command failed with exit code 1: (cd client && npm run build)
8:27:30 AM: ​
8:27:30 AM: Error location
8:27:30 AM: In build.command from netlify.toml:
8:27:30 AM: (cd client && npm run build)
8:27:30 AM: ​
8:27:30 AM: Resolved config
8:27:30 AM: build:
8:27:30 AM: command: (cd client && npm run build)
8:27:30 AM: commandOrigin: config
8:27:30 AM: functions: /opt/build/repo/serverless
8:27:30 AM: publish: /opt/build/repo/client/public
8:27:30 AM: headers:
8:27:30 AM: - for: ‘.js’
8:27:30 AM: values:
8:27:30 AM: Cache-Control: public, max-age=604800
8:27:30 AM: - for: '
.css’
8:27:30 AM: values:
8:27:30 AM: Cache-Control: public, max-age=604800
8:27:30 AM: plugins:
8:27:30 AM: - inputs: {}
8:27:30 AM: origin: config
8:27:30 AM: package: ‘@netlify/plugin-lighthouse’
8:27:30 AM: - inputs: {}
8:27:30 AM: origin: config
8:27:30 AM: package: netlify-plugin-gatsby-cache
8:27:30 AM: - inputs: {}
8:27:30 AM: origin: config
8:27:30 AM: package: netlify-plugin-inline-critical-css
8:27:30 AM: Caching artifacts
8:27:30 AM: Started saving build plugins
8:27:30 AM: Finished saving build plugins
8:27:30 AM: Started saving pip cache
8:27:30 AM: Finished saving pip cache
8:27:30 AM: Started saving emacs cask dependencies
8:27:30 AM: Finished saving emacs cask dependencies
8:27:30 AM: Started saving maven dependencies
8:27:30 AM: Finished saving maven dependencies
8:27:30 AM: Started saving boot dependencies
8:27:30 AM: Finished saving boot dependencies
8:27:30 AM: Started saving rust rustup cache
8:27:30 AM: Finished saving rust rustup cache
8:27:30 AM: Started saving go dependencies
8:27:30 AM: Finished saving go dependencies
8:27:33 AM: Build failed due to a user error: Build script returned non-zero exit code: 2
8:27:33 AM: Failing build: Failed to build site
8:27:33 AM: Failed during stage ‘building site’: Build script returned non-zero exit code: 2
8:27:33 AM: Finished processing build request in 1m41.958045998s

Thanks for the repo. I’ll check this in a while.

Oh wait, I just checked your netlify.toml and you have your command with (). Does removing it make a difference? I mean, try with command = "cd client && npm run build"

I’ve tried both and neither work. I’ve update the repo and will post the netlify build here once its done.

Here’s the netlify build without parenthesis (note same error message):

9:31:17 AM: Build ready to start
9:31:19 AM: build-image version: d84c79427e8f83c1ba17bcdd7b3fe38059376b68
9:31:19 AM: build-image tag: v3.6.1
9:31:19 AM: buildbot version: df42d44cc6a0381cf657a671baf049c03981bafe
9:31:19 AM: Fetching cached dependencies
9:31:19 AM: Failed to fetch cache, continuing with build
9:31:19 AM: Starting to prepare the repo for build
9:31:19 AM: No cached dependencies found. Cloning fresh repo
9:31:19 AM: git clone https://github.com/LuisOsta/netlify-test
9:31:20 AM: Preparing Git Reference refs/heads/master
9:31:21 AM: Different publish path detected, going to use the one specified in the Netlify configuration file: 'client/public' versus '/' in the Netlify UI
9:31:21 AM: Different functions path detected, going to use the one specified in the Netlify configuration file: 'serverless' versus '' in the Netlify UI
9:31:21 AM: Different build command detected, going to use the one specified in the Netlify configuration file: 'cd client && npm run build' versus '' in the Netlify UI
9:31:21 AM: Starting build script
9:31:21 AM: Installing dependencies
9:31:21 AM: Python version set to 2.7
9:31:22 AM: v12.18.0 is already installed.
9:31:23 AM: Now using node v12.18.0 (npm v6.14.4)
9:31:23 AM: Started restoring cached build plugins
9:31:23 AM: Finished restoring cached build plugins
9:31:23 AM: Attempting ruby version 2.7.1, read from environment
9:31:24 AM: Using ruby version 2.7.1
9:31:24 AM: Using PHP version 5.6
9:31:24 AM: Started restoring cached go cache
9:31:24 AM: Finished restoring cached go cache
9:31:24 AM: go version go1.14.4 linux/amd64
9:31:24 AM: go version go1.14.4 linux/amd64
9:31:24 AM: Installing missing commands
9:31:24 AM: Verify run directory
9:31:25 AM: ​
9:31:25 AM: ────────────────────────────────────────────────────────────────
9:31:25 AM:   Netlify Build                                                 
9:31:25 AM: ────────────────────────────────────────────────────────────────
9:31:25 AM: ​
9:31:25 AM: ❯ Version
9:31:25 AM:   @netlify/build 9.1.3
9:31:25 AM: ​
9:31:25 AM: ❯ Flags
9:31:25 AM:   deployId: 6033ce45328820000703abb4
9:31:25 AM:   mode: buildbot
9:31:25 AM: ​
9:31:25 AM: ❯ Current directory
9:31:25 AM:   /opt/build/repo
9:31:25 AM: ​
9:31:25 AM: ❯ Config file
9:31:25 AM:   /opt/build/repo/netlify.toml
9:31:25 AM: ​
9:31:25 AM: ❯ Context
9:31:25 AM:   production
9:31:25 AM: ​
9:31:25 AM: ❯ Installing plugins
9:31:25 AM:    - @netlify/plugin-lighthouse@1.4.2
9:31:25 AM:    - netlify-plugin-gatsby-cache@0.3.0
9:31:25 AM:    - netlify-plugin-inline-critical-css@1.2.0
9:32:15 AM: ​
9:32:15 AM: ❯ Loading plugins
9:32:15 AM:    - @netlify/plugin-lighthouse@1.4.2 from netlify.toml
9:32:15 AM:    - netlify-plugin-gatsby-cache@0.3.0 from netlify.toml
9:32:15 AM:    - netlify-plugin-inline-critical-css@1.2.0 from netlify.toml
9:32:16 AM: ​
9:32:16 AM: ────────────────────────────────────────────────────────────────
9:32:16 AM:   1. onPreBuild command from netlify-plugin-gatsby-cache        
9:32:16 AM: ────────────────────────────────────────────────────────────────
9:32:16 AM: ​
9:32:16 AM: No Gatsby cache found. Building fresh.
9:32:16 AM: ​
9:32:16 AM: (netlify-plugin-gatsby-cache onPreBuild completed in 166ms)
9:32:16 AM: ​
9:32:16 AM: ────────────────────────────────────────────────────────────────
9:32:16 AM:   2. build.command from netlify.toml                            
9:32:16 AM: ────────────────────────────────────────────────────────────────
9:32:16 AM: ​
9:32:16 AM: $ cd client && npm run build
9:32:17 AM: > client@0.1.0 build /opt/build/repo/client
9:32:17 AM: > react-scripts build
9:32:17 AM: sh: 1: react-scripts: not found
9:32:17 AM: npm ERR! code ELIFECYCLE
9:32:17 AM: npm ERR! syscall spawn
9:32:17 AM: npm ERR! file sh
9:32:17 AM: npm ERR! errno ENOENT
9:32:17 AM: npm ERR! client@0.1.0 build: `react-scripts build`
9:32:17 AM: npm ERR! spawn ENOENT
9:32:17 AM: npm ERR!
9:32:17 AM: npm ERR! Failed at the client@0.1.0 build script.
9:32:17 AM: npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
9:32:17 AM: npm WARN Local package.json exists, but node_modules missing, did you mean to install?
9:32:17 AM: npm ERR! A complete log of this run can be found in:
9:32:17 AM: npm ERR!     /opt/buildhome/.npm/_logs/2021-02-22T15_32_17_052Z-debug.log
9:32:17 AM: ​
9:32:17 AM: ────────────────────────────────────────────────────────────────
9:32:17 AM:   "build.command" failed                                        
9:32:17 AM: ────────────────────────────────────────────────────────────────
9:32:17 AM: ​
9:32:17 AM:   Error message
9:32:17 AM:   Command failed with exit code 1: cd client && npm run build
9:32:17 AM: ​
9:32:17 AM:   Error location
9:32:17 AM:   In build.command from netlify.toml:
9:32:17 AM:   cd client && npm run build
9:32:17 AM: ​
9:32:17 AM:   Resolved config
9:32:17 AM:   build:
9:32:17 AM:     command: cd client && npm run build
9:32:17 AM:     commandOrigin: config
9:32:17 AM:     functions: /opt/build/repo/serverless
9:32:17 AM:     publish: /opt/build/repo/client/public
9:32:17 AM:   headers:
9:32:17 AM:     - for: '*.js'
9:32:17 AM:       values:
9:32:17 AM:         Cache-Control: public, max-age=604800
9:32:17 AM:     - for: '*.css'
9:32:17 AM:       values:
9:32:17 AM:         Cache-Control: public, max-age=604800
9:32:17 AM:   plugins:
9:32:17 AM:     - inputs: {}
9:32:17 AM:       origin: config
9:32:17 AM:       package: '@netlify/plugin-lighthouse'
9:32:17 AM:     - inputs: {}
9:32:17 AM:       origin: config
9:32:17 AM:       package: netlify-plugin-gatsby-cache
9:32:17 AM:     - inputs: {}
9:32:17 AM:       origin: config
9:32:17 AM:       package: netlify-plugin-inline-critical-css
9:32:17 AM: Caching artifacts
9:32:17 AM: Started saving build plugins
9:32:17 AM: Finished saving build plugins
9:32:17 AM: Started saving pip cache
9:32:17 AM: Finished saving pip cache
9:32:17 AM: Started saving emacs cask dependencies
9:32:17 AM: Finished saving emacs cask dependencies
9:32:17 AM: Started saving maven dependencies
9:32:17 AM: Finished saving maven dependencies
9:32:17 AM: Started saving boot dependencies
9:32:17 AM: Finished saving boot dependencies
9:32:17 AM: Started saving rust rustup cache
9:32:17 AM: Finished saving rust rustup cache
9:32:17 AM: Started saving go dependencies
9:32:17 AM: Finished saving go dependencies
9:32:19 AM: Build failed due to a user error: Build script returned non-zero exit code: 2
9:32:19 AM: Failed during stage 'building site': Build script returned non-zero exit code: 2
9:32:19 AM: Failing build: Failed to build site
9:32:19 AM: Finished processing build request in 1m0.172395796s

Alright. I’ll get into this.

1 Like

Thanks I really appreciate the help! Hopefully somebody from the Netlify team can also hop in and help as well.

I have a feeling it may be tied to how they handle root directories. But I’m really hoping we can find a good solution

Netlify won’t magically do an install in your client folder so you need to put npm install before npm run build. I run a split setup in one of my projects but I did it using a package.json in the root (which pretty much only installs netlify-cli so I can use yarn dev when developing). From there I run yarn in different sub directories with yarn --cwd (via run-p to run in parallel).

Hope it sheds some light on the problem.

1 Like

Thanks for the tip! What do you mean by “split setup”?

Also, can you share any other insights you had making your setup works?

So if anyone is reading this thread wanting to figure out how to get it working. This repo is now working - GitHub - LuisOsta/netlify-test. With both the functions and build functioning

1 Like

Sure, this is my root package.json:

{
  "scripts": {
    "dev": "netlify dev",
    "build": "yarn packages:utils && yarn packages:lambda && yarn packages:www && run-p build:**",
    "packages:utils": "yarn --cwd src/utils",
    "packages:lambda": "yarn --cwd src/lambda",
    "packages:www": "yarn install --cwd src/www --network-concurrency 1",
    "build:www": "yarn --cwd src/www build"
  },
  "dependencies": {
    "netlify-cli": "^3.8.1",
    "npm-run-all": "^4.1.5"
  }
}

And src/www/package.json:

{
  "scripts": {
    "dev": "webpack serve",
    "build": "webpack --mode production"
  },
  [...]
}

The other two basically only have dependencies.

I am in no way saying that this exact config is the way to go though, I am still experimenting. I was experiencing some problems with the cache, that’s why I chose to run some things in serial (mostly due to lack of debug time at the moment).

This is just what fit my current workflow, I can now run yarn dev in the root and start the netlify environment.

1 Like

Yeah, I think I’m going to start diving deeper into it as well. To try to figure out a good flow. Ideally, I’d love to get the Netlify functions working nicely with the Serverless Framework but that might be a bit of a pipe dream.

Good news! I wish everyone came back and updated with a working example like this! Thanks for making this community even better! :clinking_glasses: :ok_hand:

2 Likes

Haha you’re too kind! Wouldn’t have even found the answer if it wasn’t for you.

3 Likes