Mac OS X handles trailing slashes in cp build command differently than build image

[danielthedifficult]

I’m having a problem:
netlify dev works great for my files + api.
Running netlify build && netlify deploy (with or without --prod) works great.

Triggering a deploy with a new commit results in getting 404’s when I visit my site.

I downloaded the .zip of the deployment, and it only has the netlify.toml at the root, and a ./src folder with my source code… but I don’t know how that’s happening.

It should be deploying from a ./dist folder, per my build commands, and the configuration I’ve specified in netlify.toml.

After a GitHub/automated deployment:
Visiting my.site.com/src/ does work.
Visiting my.site.com 404’s.

After local build & deploy:
Visiting my.site.com works.

Questions:

Why is netlify building to a ./src folder when all my configuration for production deployments is supposed to build to ./dist?

Why does building + publishing from the CLI work, but building via your build system exhibits this behavior?

And how can we fix it??

Thank you

My repo’s file structure:
apidoc.json
docs
netlify.toml
package.json
yarn.lock
dist/ ← My build destination
jest.env
node_modules/
src/ ← Where all my code files live

Here is my Netlify.toml:

[dev]
# This is where my code lives, so when I test locally with 'netlify dev' I load the 'live' code i'm working on. This works great.
  publish = "src/"

[build]
  base = "/"

# This is the directory I create during my build step
  publish = "dist/"

  # Default build command.
# See build log for details, but I basically copy my files from /src to /dist and delete certain things I don't want hosted
  command = "yarn build"

# Added the below thinking it would help. It didn't.
[context.production]
  publish = "dist/"
[context.deploy-preview]
  publish = "dist/"

[[redirects]]
  from = "/api/*"
  to = "/.netlify/functions/:splat"
  status = 200
[[redirects]]
  from = "/*"
  to = "/index.html"
  status = 200

[[headers]]
  for = "/*"

  [headers.values]
    X-Frame-Options = "DENY"
    X-XSS-Protection = "1; mode=block"
    Content-Security-Policy = "frame-ancestors https://www.facebook.com"

    # Multi-value headers are expressed with multi-line strings.
	cache-control = '''
	max-age=0,
	no-cache,
	no-store,
	must-revalidate'''


[functions]
  directory = "src/api/"

And here is the build Log:

12:52:07 PM: Build ready to start
12:52:10 PM: build-image version: 195fbe127e5c374d9c4758652cb62e3b8936a395 (focal)
12:52:10 PM: build-image tag: v4.6.0
12:52:10 PM: buildbot version: 4ac77ee15240cafa5a14a04b6ca18d8959fa98e6
12:52:10 PM: Fetching cached dependencies
12:52:10 PM: Starting to download cache of 487.6MB
12:52:15 PM: Finished downloading cache in 5.136922933s
12:52:15 PM: Starting to extract cache
12:52:27 PM: Finished extracting cache in 11.6214067s
12:52:27 PM: Finished fetching cache in 16.828497922s
12:52:27 PM: Starting to prepare the repo for build
12:52:27 PM: Preparing Git Reference refs/heads/prepare-for-netlify
12:52:30 PM: Parsing package.json dependencies
12:52:30 PM: Different functions path detected, going to use the one specified in the Netlify configuration file: 'src/api' versus '' in the Netlify UI
12:52:31 PM: Starting build script
12:52:31 PM: Installing dependencies
12:52:31 PM: Python version set to 2.7
12:52:31 PM: Started restoring cached node version
12:52:32 PM: Finished restoring cached node version
12:52:33 PM: v16.14.1 is already installed.
12:52:33 PM: Now using node v16.14.1 (npm v8.5.0)
12:52:33 PM: Started restoring cached build plugins
12:52:33 PM: Finished restoring cached build plugins
12:52:33 PM: Attempting ruby version 2.7.2, read from environment
12:52:34 PM: Using ruby version 2.7.2
12:52:34 PM: Using PHP version 8.0
12:52:34 PM: Started restoring cached yarn cache
12:52:36 PM: Finished restoring cached yarn cache
12:52:36 PM: No yarn workspaces detected
12:52:36 PM: Started restoring cached node modules
12:52:36 PM: Finished restoring cached node modules
12:52:36 PM: Installing NPM modules using Yarn version 1.22.10
12:52:37 PM: yarn install v1.22.10
12:52:37 PM: [1/4] Resolving packages...
12:52:37 PM: success Already up-to-date.
12:52:37 PM: Done in 0.25s.
12:52:37 PM: NPM modules installed using Yarn
12:52:37 PM: Started restoring cached go cache
12:52:37 PM: Finished restoring cached go cache
12:52:37 PM: go version go1.16.5 linux/amd64
12:52:37 PM: go version go1.16.5 linux/amd64
12:52:37 PM: Installing missing commands
12:52:37 PM: Verify run directory
12:52:39 PM: ​
12:52:39 PM: ────────────────────────────────────────────────────────────────
12:52:39 PM:   Netlify Build                                                 
12:52:39 PM: ────────────────────────────────────────────────────────────────
12:52:39 PM: ​
12:52:39 PM: ❯ Version
12:52:39 PM:   @netlify/build 26.5.0
12:52:39 PM: ​
12:52:39 PM: ❯ Flags
12:52:39 PM:   baseRelDir: true
12:52:39 PM:   buildId: 623320e7c0540e0008d878a6
12:52:39 PM:   deployId: 623320e7c0540e0008d878a8
12:52:39 PM: ​
12:52:39 PM: ❯ Current directory
12:52:39 PM:   /opt/build/repo
12:52:39 PM: ​
12:52:39 PM: ❯ Config file
12:52:39 PM:   /opt/build/repo/netlify.toml
12:52:39 PM: ​
12:52:39 PM: ❯ Context
12:52:39 PM:   production
12:52:39 PM: ​
12:52:39 PM: ────────────────────────────────────────────────────────────────
12:52:39 PM:   1. build.command from netlify.toml                            
12:52:39 PM: ────────────────────────────────────────────────────────────────
12:52:39 PM: ​
12:52:39 PM: $ yarn build
12:52:39 PM: yarn run v1.22.10
12:52:39 PM: $ rm -rf ./dist && mkdir ./dist
12:52:39 PM: $ cp -R ./src/ ./dist
12:52:39 PM: $ cd ./dist && rm -rf some_folders_I_want_to_delete_from_public_build
12:52:39 PM: Done in 0.12s.
12:52:39 PM: ​
12:52:39 PM: (build.command completed in 302ms)
12:52:39 PM: ​
12:52:39 PM: ────────────────────────────────────────────────────────────────
12:52:39 PM:   2. Functions bundling                                         
12:52:39 PM: ────────────────────────────────────────────────────────────────
12:52:39 PM: ​
12:52:39 PM: Packaging Functions from src/api directory:
**[redacted, but functions are deploying correctly]**
12:52:39 PM: ​
12:52:53 PM: ​
12:52:53 PM: (Functions bundling completed in 14.1s)
12:52:53 PM: ​
12:52:53 PM: ────────────────────────────────────────────────────────────────
12:52:53 PM:   3. Deploy site                                                
12:52:53 PM: ────────────────────────────────────────────────────────────────
12:52:53 PM: ​
12:52:53 PM: Starting to deploy site from 'dist'
12:52:54 PM: Creating deploy tree asynchronously
12:52:54 PM: Creating deploy upload records
12:52:54 PM: Starting post processing
12:52:55 PM: Post processing - HTML
12:52:55 PM: Post processing - header rules
12:52:55 PM: Post processing - redirect rules
12:52:55 PM: Post processing done
12:52:59 PM: Site is live ✨
12:53:00 PM: 0 new files to upload
12:53:00 PM: 0 new functions to upload
12:53:00 PM: Site deploy was successfully initiated
12:53:00 PM: ​
12:53:00 PM: (Deploy site completed in 6.6s)
12:53:00 PM: ​
12:53:00 PM: ────────────────────────────────────────────────────────────────
12:53:00 PM:   Netlify Build Complete                                        
12:53:00 PM: ────────────────────────────────────────────────────────────────
12:53:00 PM: ​
12:53:00 PM: (Netlify Build completed in 21.2s)
12:53:00 PM: Caching artifacts
12:53:00 PM: Started saving node modules
12:53:00 PM: Finished saving node modules
12:53:00 PM: Started saving build plugins
12:53:00 PM: Finished saving build plugins
12:53:00 PM: Started saving yarn cache
12:53:03 PM: Finished saving yarn cache
12:53:03 PM: Started saving pip cache
12:53:03 PM: Finished saving pip cache
12:53:03 PM: Started saving emacs cask dependencies
12:53:03 PM: Finished saving emacs cask dependencies
12:53:03 PM: Started saving maven dependencies
12:53:03 PM: Finished saving maven dependencies
12:53:03 PM: Started saving boot dependencies
12:53:03 PM: Finished saving boot dependencies
12:53:03 PM: Started saving rust rustup cache
12:53:03 PM: Finished saving rust rustup cache
12:53:03 PM: Started saving go dependencies
12:53:03 PM: Finished saving go dependencies
12:53:03 PM: Build script success
12:54:09 PM: Finished processing build request in 1m59.325235144s

[hrishikesh]

Hey @danielthedifficult,

Netlify is not using dev context, rather it’s doing as you’re asking it to do.

Your entire site exists correctly inside the https://your-site.netlify.app/src/ folder. You’re copying the src folder into the dist. That’s not what you want. You can do what you wish to using shell scripts like explained here: command line - Recursively copy the contents of subfolders - Ask Ubuntu. But an easier way would be to simply run rm -rf inside your src folder and publish that without copying it anywhere. Running rm -rf on Netlify won’t affect the files in your repo.

[danielthedifficult]

Hello Hrishikesh,

Can you please help me understand why netlify is building my site inside a /src/ folder?

You did say that I’m copying the src folder into the dist, but the published site (as downloaded from Netlify after build) does not show a dist folder at all. The way netlify builds it via your servers, I can’t find a /dist folder anywhere.

I don’t want to add the rm -rf ./src commands to my build process, because if I build locally, I could end up deleting files out of my (local) /src folder. Yes, I could check env vars, etc., but we only need one edge case to start deleting files locally, and I want to keep things simple.

The purpose of using the /dist folder is to copy only some of the files from src and publish them.

1. Where is the ./src folder coming from in my finalized build? Is that a netlify default? Or is it getting it from my local folder structure? The dev.context setting?
2. I’m repeating my question from my original post: why does it build and run locally from dist, but ends up publishing to ./src when netlify builds it?

[hrishikesh]

@danielthedifficult

You did say that I’m copying the src folder into the dist , but the published site (as downloaded from Netlify after build) does not show a dist folder at all. The way netlify builds it via your servers, I can’t find a /dist folder anywhere.

The contents of the dist folder are deployed. Thus you can’t see the dist folder (as that’s the parent folder to the files you’re seeing). Your dist folder contains your src folder (based on your build command). You’re not copying the contents of src folder, you’re copying the src folder as it is into the dist, which is why, your website is available on the src subdirectory.

[danielthedifficult]

Oh, that makes total sense. I’m going to dig in and figure out why it behaves differently locally vs. built on your platform, it may be a nuance of how one of the commands handles trailing slashes.

Thank you!

[danielthedifficult]

Figured it out!

I am developing on OS X / Darwin kernel, and trailing slashes are handled differently than on Linux.

This gave me a hint:

Recursive cp copies folder contents instead of folder on OS X

I changed my build command from:
cp -r ./src/ ./dist
to:
cp -r ./src/* ./dist

And now builds work both locally and on the Ubuntu 20.04 image

Thanks for the help! Understanding that what I was downloading was the .dist folder was key.