Possibility to ignore git submodules

[tobiasdiez]

I have a Git repository linked to a Netlify site. The repository includes several Git submodules, but these submodules are not required for building or deploying the website. (It’s a monorepo, and the website is only a part of it.)

The issue is that the submodules point to private Git repositories, so Netlify’s checkout process fails because it doesn’t have the required permissions. Unfortunately, I can’t add the necessary credentials to Netlify to allow it to access these submodules.

Is there a way to prevent Netlify from trying to checkout the submodules, or at least configure it to continue the build even if the submodules aren’t available?

[sid.m]

The ignore command might be a solution in this case:

Ignore builds

[Support Guide] How to use the ignore command

Last Reviewed By Netlify Support Staff: November 2024 The ignore command is a powerful way for you to control whether or not builds run based on variables of your choosing. Here’s how it works: you can set ignore in your netlify.toml to any Bash command or Bash script. If your ignore command or script exits 0, the build is canceled. If the command or script exits with 1 or any other exit code, the build continues. We created this feature for monorepo support, but it is useful in other context…

Let us know if that’s a feasible solution for you!

[tobiasdiez]

Thanks for the suggestion. How does this ignore command helps? The build already fails at the git-checkout stage since it’s not able to checkout the submodules.

[Melvin]

I don’t believe it’s possible to ignore a submodule unless the standard npm install command has a way to ignore it. This is our documentation on git submodules:

Repository permissions and linking

We also support npm flags that you can pass during installation time:

Build environment variables

[tobiasdiez]

Thanks, but the problem is that it fails already at checkout and thus it is not possible to run any build commands. I couldn’t find any option to customize the checkout command. Is there one?

[clarnx]

Hi @tobiasdiez, thanks for the extra feedback.

The main issue you’re facing is that Netlify’s build process attempts to fetch all submodules, including those that are private and not necessary for your website build. This can cause problems during the build process.

Kindly try out the suggestions below to see if it works.

1. You can configure Netlify to ignore certain directories or files during the build process. This can be done using the ignore builds feature.
2. If your website is in a specific subdirectory of your monorepo, you can configure Netlify to only build from that directory. Check the documentation at Build configuration overview
3. Manually configure submodules by following the steps provided in the Support Guide: How do I access private repositories in the build environment?

If none of the above steps worked consider restructuring your repository to separate the website part from the rest of the monorepo if possible. This could simplify your Netlify deployment process.

Any changes to your repository structure or build configuration might require updates to your build settings in Netlify. You can adjust these in your site’s dashboard under Site configuration > Build & deploy > Continuous deployment > Build settings .

Kindly write back and let us know which of the methods worked for you.

Thanks.

[tobiasdiez]

Thanks a lot for the reply.

The first two options are not working because they these build scripts/ignore statements are only considered for the actual build, but the checkout of the repo happens before.

It already fails in the initialization step with something of this form:

4:54:55 PM: Build ready to start
4:55:11 PM: build-image version: 92f088c58b398b4ea0fe23a49753d86a87399079 (focal)
4:55:11 PM: buildbot version: 92f088c58b398b4ea0fe23a49753d86a87399079
4:55:11 PM: Fetching cached dependencies
4:55:11 PM: Failed to fetch cache, continuing with build
4:55:11 PM: Starting to prepare the repo for build
4:55:12 PM: No cached dependencies found. Cloning fresh repo
4:55:12 PM: git clone --filter=blob:none <main repo>
4:55:12 PM: Preparing Git Reference refs/heads/main
4:55:26 PM: Error checking out submodules: Submodule '...'
Submodule '<submodule>'
Cloning into '/opt/build/repo/<submodule>'...
fatal: could not read Username for 'https://github.com': No such device or address
fatal: clone of '<submodule>' failed

The checks for custom build scripts/ignore only comes after checking out the submodules, which is too late.

Option 3 may work for some people, but I don’t have the rights (and the possibility) to add deploy keys for the submodule repositories.

[clarnx]

Hi @tobiasdiez, Option 3 would have helped. However, since you mentioned that you don’t have the rights and the possibility to add deploy keys, then that’s beyond the scope of help of the forums.
I apologize for the inconvenience.
Thanks.

[tobiasdiez]

Thanks for the confirmation that this is currently not possible.

Could you then please escalate this issue to a feature request? In my case, a simple toggle to checkout submodules would be sufficient (perhaps other people would need a more fine-grained control).

[clarnx]

Hi @tobiasdiez,
Thank you for reaching out and sharing your feature request! We truly appreciate your feedback and suggestions.

At the moment, since this feature is not available, and while we don’t have an immediate timeline for its release, we’re always working on improving our product.

Please let us know if there’s anything else we can assist you with!
Thanks.