Custom build ignore script exit code bug

hkgumbs · November 28, 2019, 12:04pm

I am trying to understand and use Netlify’s monorepo support. I cannot produce the build.ignore behavior described in the docs:

An exit-code of 0 indicates that the build should return early.

I’ve setup a PR on my repo to test this with the simplest script possible: exit 0. Here is a snippet from the logs, where you can see that the build continues after running the build.ignore command.

6:35:13 AM: Found Netlify configuration file. Overriding site configuration
6:35:13 AM: Detected ignore command in Netlify configuration file. Proceeding with the specified command: 'exit 0'
6:35:13 AM: Starting build script

Netlify instance names: trainervision.netlify.com and trainervision-amy.netlify.com

luke · December 2, 2019, 8:52am

I think the exit 0 is exiting the sub-shell and that is why it isn’t working. Would you try making a script which does this instead of running exit 0 directly? For example:

#!/bin/bash

exit 0

This might be saved to a file named exit-zero.sh (with a chmod permission of 700). The build.ignore command would then be the filename “exit-zero.sh” instead.

If this doesn’t work as expected, please let us know.

hkgumbs · December 2, 2019, 4:38pm

I’m still seeing the behavior with the exit-zero.sh script change:

11:24:13 AM: Detected ignore command in Netlify configuration file. Proceeding with the specified command: 'exit-zero.sh'
11:24:13 AM: Starting build script

Here’s the link to the full build log for that site (trainervision-amy.netlify.com): https://app.netlify.com/sites/trainervision-amy/deploys/5de53aa1aba2ad00083d981d

Also, the other project I’m trying to deploy from this directory (trainervision.netlify.com) is also ignoring the ignore script, but in that case there is no reference of “ignore” anywhere in the logs at all. I bring up the other project because in my PR, I added the script in both the root directory and the subproject directory to see if it either project deployed correctly. Neither did. I’ve also un-linked/re-linked both projects to the GitHub repo.

luke · December 5, 2019, 9:03am

@hkgumbs, I did some testing and I can reproduce this behavior if the path to the build.ignore script doesn’t match the filesystem location.

I found two root causes:

If the path to the command/script in build.ignore is invalid, the build continues silently assuming a build should continue.
If running a script (and not a command like git diff), the script must be referenced relative to the base directory of the repo (or, possibly, the build directory - I didn’t test that edge case).

So, instead of this:

  ignore = "exit-zero.sh"

Try this instead:

  ignore = "./exit-zero.sh"

I’ll be filing an issue to about this first root cause above and, instead, have the build system log when there is missing/invalid path for build.ignore. I’ll update this topic once more when this issue is filed.

The second root cause isn’t is a bug, technically. It’s default behavior in a bash shell, but I also believe we should have better documentation about this syntax requirement for scripts in our docs.

So, I have two issues to file and I follow-up here again to let you know when that is done (and to cross-link this topic with the issues).

In the meantime, would you please test adding “./” before any script reference and double-check the path to the script is correct? If that doesn’t solve the issue, please let us know.

luke · December 6, 2019, 9:44am

I filed this issue to get the build-image’s behavior changed regarding missing scripts/commands:

github.com/netlify/build-image

When the build.ignore command or script is not found, log this to the build logs.

opened 09:15AM - 06 Dec 19 UTC

overlordofmu

**Description** If you reference a `build.ignore` which is invalid or doesn't… exist the failure is silently ignored and the build continues. I was testing with a bash script named `exit-zero.sh`: ``` #!/bin/bash exit 0 ``` If I reference this in `build.ignore` as `./exit-zero.sh` the script runs, exits with zero and the build is cancelled: ``` 11:59:22 PM: Canceled build due to no content change 11:59:22 PM: Detected ignore command in Netlify configuration file. Proceeding with the specified command: './exit-zero.sh' 11:59:22 PM: User-specified ignore command returned exit code 0. Returning early from build. ``` However, if I don't reference this from the current directory it isn't in the shell's path so cannot be run at all. However, the logging makes it look no different than any other non-zero exit: ``` 12:57:42 AM: Detected ignore command in Netlify configuration file. Proceeding with the specified command: 'exit-zero.sh' 12:57:42 AM: No build command found, continuing to publishing 12:57:42 AM: Starting to deploy site from '/' ``` **Solution I would like** Ideally, the build would still continue (as this should be the default I believe) but we would log the missing command or script. For example: ``` 12:57:42 AM: Detected ignore command in Netlify configuration file. Proceeding with the specified command: 'exit-zero.sh' 12:57:42 AM: exit-zero.sh: command not found 12:57:42 AM: No build command found, continuing to publishing 12:57:42 AM: Starting to deploy site from '/' ``` **Other notes** This could be a simple as filtering for "command not found" and logging if that string is in the output for the `build.ignore` attempt. **Follow-up** Notify here: https://community.netlify.com/t/custom-build-ignore-script-exit-code-bug/5627/3

We’ll update this topic again if/when that issue is resolved.

hkgumbs · December 6, 2019, 5:55pm

Thanks Luke, that worked for the subdirectory site! I’m still having trouble getting the main site (trainervision) to notice the build.ignore flag. I’ve relinked the site to my GitHub repo, but I suspect it’s still not using the new setup. Can you see on your side whether trainervision.netlify.com is setup to support build.ignore? Here’s what I see in the logs where the ignore check typically is:

12:47:21 PM: Preparing Git Reference pull/5/head
12:47:22 PM: Found netlify.toml. Overriding site configuration
12:47:22 PM: Starting build script

luke · December 9, 2019, 7:27am

Hi, @hkgumbs, I don’t show netlify.toml being processed in the most recent published deploy.

Would you please make certain there is netlify.toml file base of the repo (as opposed to the publish directory)?

If there is, would you please either post that file here (with all sensitive details redacted) and/or send it to me in a private message (PM)?

hkgumbs · December 9, 2019, 6:10pm

The deploy you linked to is from my master branch, but I’ve been experimenting with these settings on the branch called netlify-ignore. To clarify things, I pushed an update to that branch this morning, which seems to clearly illustrate the issue:

The “subdirectory site” deploy, which correctly picks up `buld.ignore` from netlify.toml

The “main site” deploy, does NOT pick up the `build.ignore`

The full PR diff

I added a single netlify.toml file in the root directory, a copy of the exit-zero.sh script in the root directory, and a copy of the exit-zero.sh file in the subdirectory (to make sure it would be picked up by both sites).

diff --git a/exit-zero.sh b/exit-zero.sh
new file mode 100755
index 0000000..742e13d
--- /dev/null
+++ b/exit-zero.sh
@@ -0,0 +1,2 @@
+#!/usr/bin/env bash
+exit 0
diff --git a/netlify.toml b/netlify.toml
new file mode 100644
index 0000000..390fb61
--- /dev/null
+++ b/netlify.toml
@@ -0,0 +1,2 @@
+[build]
+  ignore = "./exit-zero.sh"
diff --git a/sites/amy/exit-zero.sh b/sites/amy/exit-zero.sh
new file mode 100755
index 0000000..742e13d
--- /dev/null
+++ b/sites/amy/exit-zero.sh
@@ -0,0 +1,2 @@
+#!/usr/bin/env bash
+exit 0

fool · December 15, 2019, 4:55pm

Sorry to be so slow to get back to you. Still not sure what is happening exactly. Could it be permissions based? Git and any specific file permissions is pretty hard to get reliably correct - could you try modifying to sh ./exit-zero.sh instead of relying on it to be executable after clone?

hkgumbs · December 15, 2019, 7:19pm

Unfortunately, that did not seem to help things. The subdirectory site deployed OK, but the main site still does not detect any build.ignore settings: Netlify App.

This site is older than the subdirectory site, so I suspect it’s something to do with the unlinking/relinking on the main site. But I don’t know how to tell what build-image version I’m using.

fool · December 17, 2019, 4:50am

ah, mystery solved! You use our super-legacy trusty build image so your site is doubly too old to use the feature :). You would need to:

update to xenial, here: Netlify App
re-link your repo, here: Netlify App

hkgumbs · December 17, 2019, 6:00pm

That worked! Thanks for getting to the bottom of it!

itswadesh · April 5, 2020, 6:21pm

Hi,

Sorry for reopening this, but I have a monorepo setup. There I am trying to trigger build manually.

It says
11:38:13 PM: Detected ignore command in Netlify configuration file. Proceeding with the specified command: './exit-zero.sh' 11:38:13 PM: Starting build script

Here is my exit-zero.sh

` #!/bin/bash

exit 0 `

How to stop auto building?

apuntovanini · April 8, 2020, 1:29pm

Agree, same configuration, netlify simply seems to ignore the command, and goes on:

3:25:56 PM: Detected ignore command in Netlify configuration file. Proceeding with the specified command: ‘./exit-zero.sh’
3:25:56 PM: Starting build script
3:25:56 PM: Installing dependencies

jen · April 13, 2020, 11:12pm

Hey @apuntovanini and @itswadesh, thanks for letting us know this is an ongoing issue. I was able to reproduce this and have added your reports to an issue that was opened internally about this behavior.

At this point, unfortunately, the only thing I can reliably recommend is setting ignore = "git status" in your netlify.toml file. That should exit 0 and cancel your build.

I acknowledge that this is extremely hacky and hope we’ll have more insight into what’s happening here soon!

itswadesh · April 14, 2020, 5:03am

Hello Jen,

Thanks for your time. The command git status does not seem to work for git merge. It simply ignores the deploy, even if I do retry,

it still skips build.
but downloads cache

Netlify is the only JAM stack service that supports monorepo and its working fine in my case. I have 4 websites from 1 monorepo successfully running.

The only feature I wish is: to manually select which site to build and deploy (stop netlify build it automatically)

I am a developer on free plan, so every time I push to production branch it consumes 12(4*3) minutes of build time, which is unnecessary in my case.

Can you guide?

luke · April 14, 2020, 6:07am

Hi, @itswadesh and @apuntovanini.

I think the issue is that the file permissions for the file don’t allow for the script to be run. This error then returns a non-zero exit code and that determines that a build should still occur.

For example, when I cloned this repo, I had the following results when testing:

$ ./exit-zero.sh ; echo $?
-bash: ./exit-zero.sh: Permission denied
126
$ chmod 700 ./exit-zero.sh ; ./exit-zero.sh ; echo $?
0

The first command is returning an error and the second command is returning success (0).

There is a way to confirm or disprove this hypothesis. Would you please test making the ignore command the following instead:

  ignore = "chmod 700 ./exit-zero.sh ; ./exit-zero.sh"

If that successfully prevents the build, please let us know.

itswadesh · April 15, 2020, 6:19am

Did not work

luke · April 15, 2020, 9:05am

Hi, @itswadesh, I see this in the logs:

11:26:44 PM: Detected ignore command in Netlify configuration file. Proceeding with the specified command: 'chmod 700 ./exit-zero.sh ; ./exit-zero.sh'
11:26:44 PM: chmod:
11:26:44 PM: cannot access ';'
11:26:44 PM: : No such file or directory

I think the chmod also failed based on those logs. Maybe this will work instead:

  ignore = "chmod 700 ./exit-zero.sh && ./exit-zero.sh"

itswadesh · April 22, 2020, 10:40am

Sorry luke, still not working.

Custom build ignore script exit code bug

The “subdirectory site” deploy, which correctly picks up buld.ignore from netlify.toml

The “main site” deploy, does NOT pick up the build.ignore

The full PR diff

The “subdirectory site” deploy, which correctly picks up `buld.ignore` from netlify.toml

The “main site” deploy, does NOT pick up the `build.ignore`