Permission issues when testing a build with the Docker Netlify Build Image

Cognifide · January 9, 2023, 12:19pm

Hello,

I’m trying to run my Netlify build locally, using Docker and the Netlify Build Image, with the end goal of using it to monitor and optimize memory usage during the build (related to Builds constantly killed with exit code 137)

I follow the steps described in the README.md:

I pull the image from a docker registry docker pull netlify/build:focal
I check out the build-image project at the focal branch
At the root of the newly checked-out repo, I run ./test-tools/start-image.sh path/to/site/repo to start the interactive shell
The prompt changes to buildbot@251b47429cb3:/$, indicating that the buldbot is running
In this shell, I run /opt/build-bin/build yarn assemble (as yarn assemble is the command I kick off to build my site)

The build starts but immediately runs into permission problems

buildbot@9cb9777d2bdb:/$ /opt/build-bin/build yarn assemble
npm ERR! code EACCES
npm ERR! syscall open
npm ERR! path /opt/buildhome/package-lock.json
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, open '/opt/buildhome/package-lock.json'
npm ERR!  [Error: EACCES: permission denied, open '/opt/buildhome/package-lock.json'] {
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'open',
npm ERR!   path: '/opt/buildhome/package-lock.json'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.

npm ERR! A complete log of this run can be found in:
npm ERR!     /opt/buildhome/.npm/_logs/2023-01-09T11_37_01_316Z-debug-0.log
Installing dependencies
Python version set to 3.8
Downloading and installing node v16.19.0...
Downloading https://nodejs.org/dist/v16.19.0/node-v16.19.0-linux-x64.tar.xz...

//... truncated for brevity

rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/number/virtual/to-precision.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/number/is-finite.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/number/parse-float.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/number/min-safe-integer.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/number/max-safe-integer.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/object/seal.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/object/lookup-getter.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/object/get-own-property-symbols.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/object/get-own-property-names.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/object/assign.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/object/set-prototype-of.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/object/is.js': Permission denied
rm: cannot remove '/opt/buildhome/repo/node_modules/babel-runtime/node_modules/core-js/library/fn/object/is-sealed.js': Permission denied

// ... lots of similar errors

error Could not write file "/opt/buildhome/repo/yarn-error.log": "EACCES: permission denied, open '/opt/buildhome/repo/yarn-error.log'"
error An unexpected error occurred: "EACCES: permission denied, unlink '/opt/buildhome/repo/node_modules/.yarn-integrity'".

As far as I understand, there are two main issues:

It looks like it’s complaining about files in the directory mounted from outside the container, under /opt/buildhome, which corresponds to the location of my site’s repo, checked out locally.
It cannot access /opt/buildhome/package-lock.json, which sits outside the repo folder, and is part of the image itself (or at least the setup of the buildhome directory appears to be handled in the Dockerfile).

The complete log file mentioned in the output, npm ERR! A complete log of this run can be found in: npm ERR! /opt/buildhome/.npm/_logs/2023-01-09T11_37_01_316Z-debug-0.log, does not exist once the build finishes.

Upon inspection, the current user (buldbot) doesn’t have permission to modify the package-log.json.

buildbot@28de4637e59f:~/repo$ whoami
buildbot
buildbot@28de4637e59f:~$ ls -an /opt/buildhome/
total 220
drwxr-xr-x  1 2500 2500  4096 Jan  9 11:59 .
drwxr-xr-x  1    0    0  4096 Dec  8 15:44 ..
-rw-r--r--  1 2500 2500   221 Dec  8 15:44 .bash_profile
-rw-r--r--  1 2500 2500   139 Dec  8 15:44 .bashrc
drwxr-xr-x  3 2500 2500  4096 Dec  8 15:40 .binrc
drwxr-xr-x  3 2500 2500  4096 Dec  8 15:40 .boot
drwxr-xr-x  1 2500 2500  4096 Dec  8 15:41 .cache
drwxr-xr-x  3 2500 2500  4096 Dec  8 15:44 .cargo
drwxr-xr-x 10 2500 2500  4096 Dec  8 15:41 .cask
drwxr-xr-x  3 2500 2500  4096 Dec  8 15:40 .deno
drwxr-xr-x  1 2500 2500  4096 Dec  8 15:41 .dotnet
drwxr-xr-x  3 2500 2500  4096 Dec  8 15:40 .gem
-rw-r--r--  1 1000 1000   469 Dec 14 13:00 get-build-info.mjs
drwxr-xr-x  1 2500 2500  4096 Dec  8 15:41 .gimme
drwx------  1 2500 2500  4096 Dec  8 15:40 .gnupg
drwxr-xr-x  2 2500 2500  4096 Dec  8 15:44 .homebrew-cache
drwxr-xr-x  3 2500 2500  4096 Dec  8 15:40 .lein
drwxr-xr-x  3 2500 2500  4096 Dec  8 15:40 .local
drwxr-xr-x  3 2500 2500  4096 Dec  8 15:40 .m2
-rw-r--r--  1 2500 2500   118 Dec  8 15:40 .mkshrc
drwxr-xr-x  4 2500 2500  4096 Dec  8 15:40 .npm
drwxr-xr-x  1 2500 2500  4096 Dec  8 15:40 .nvm
-rw-r--r--  1 1000 1000   857 Dec 14 13:00 package.json
-rw-r--r--  1 1000 1000 66433 Dec 14 13:00 package-lock.json
drwxr-xr-x  2 2500 2500  4096 Dec  8 15:41 .php
-rw-r--r--  1 2500 2500   338 Dec  8 15:44 .profile
drwxr-xr-x  4 2500 2500  4096 Dec  8 15:40 python2.7
lrwxrwxrwx  1 2500 2500    24 Dec  8 15:40 python2.7.18 -> /opt/buildhome/python2.7
drwxr-xr-x  4 2500 2500  4096 Dec  8 15:40 python3.8
lrwxrwxrwx  1 2500 2500    24 Dec  8 15:40 python3.8.10 -> /opt/buildhome/python3.8
drwxr-xr-x 14 1000 1000  4096 Jan  4 14:44 repo
drwxr-xr-x  2 2500 2500  4096 Dec  8 15:44 .rustup
drwxr-xr-x  1 2500 2500  4096 Dec  8 15:40 .rvm
-rw-r--r--  1 2500 2500    25 Dec  8 15:40 .rvmrc
drwxr-xr-x 10 2500 2500  4096 Dec  8 15:41 .swiftenv
drwxr-xr-x  4 2500 2500  4096 Dec  8 15:41 .templateengine
-rw-r--r--  1 2500 2500   162 Dec  8 15:41 .wget-hsts
drwxr-xr-x  4 2500 2500  4096 Dec  8 15:40 .yarn
-rw-r--r--  1 2500 2500   118 Dec  8 15:40 .zlogin
-rw-r--r--  1 2500 2500   118 Dec  8 15:40 .zshrc

Is there something wrong with the way I’m using the image? For the files mounted from my site repo, I imagine I could change the permissions manually in a pinch, though I’d much prefer to keep the permission level minimal. What about the pacakage-lock.json under /opt/buildhome and similar files?

Cognifide · January 12, 2023, 10:18am

This appears to be related to test-build has issues with access rights for folders · Issue #177 · netlify/build-image · GitHub

hrishikesh · January 15, 2023, 11:49am

Are you running macOS? And if yes, does your Terminal have full-disk access?

Cognifide · January 16, 2023, 8:56am

No, it’s not a Mac. I’m running Ubuntu 20 on WSL2, the host is Windows 10. Docker’s running in WSL.

hrishikesh · January 22, 2023, 4:33pm

Is Controlled Folder Access from Windows Defender setting somehow affecting this?

crcollver · March 13, 2023, 11:05pm

Hey @Cognifide, did you ever manage to solve this? I’m running the image in Ubuntu 22 on WSL2 with Windows 10 as my host as well and getting the same permission errors as you.

I’d prefer not to change the permissions every time I need to build.

Controlled Folder Access in Windows Defender is also off for my host machine, but the problem still occurs.

fool · March 13, 2023, 11:37pm

Hey folks! Congrats, you outwaited our build-image being deprecated! You’ll use our CLI to build instead, and it should behave better on windows than those docker instructions might have:

https://cli.netlify.com/getting-started

If you find a bug with the CLI, we’ll be much more likely find a fix, since it is still actively maintained, unlike the now out-of-date build-image repo, which we archived a couple months ago and no longer give any tech support on, since the CLI does a better job of matching the builds in our build system.

Let me know how it goes!

crcollver · March 14, 2023, 2:06am

Hey @fool! I saw the archive notice on the repository, but wasn’t sure if it was still the best way to go about profiling resource utilization, since this post mentioned it: [Support Guide] Making sure your builds use appropriate resources for Netlify's build system.

Are there any suggestions for monitoring resource utilization? There seems to be a debug flag for the build command, but I don’t see any other options for monitoring. I know the Gatsby article mentions using the process-top package, but I haven’t had much luck completing a build with it when using NextJS.

Thank you for the help!

amelia · March 14, 2023, 6:39pm

Hey @crcollver , if you’re not getting the info you’re looking for when you use the debug flag, you might try the status command as well. Let us know if that helps or if you still have questions!

crcollver · March 15, 2023, 5:42pm

Hi @amelia, unfortunately the status command isn’t exactly what I was looking for either, but it is helpful! I think I need to do a bit more research on monitoring my resource utilization for a NextJS build process. I’ll report back if I manage to figure that out!