Deploy failure due to inability to clone theme submodule

Support
, ,
1

Context: Production master@HEAD
Deployment Type: Deployment with Git
Last Broken Deploy URL: Netlify App
Last Successful Deploy URL: Netlify App

Site name: ovolosash.netlify.app
Custom domain: https://www.ovolosash.com

My site is built with the Hugo SSG and its repository hosted on Gitlab. I have two themes added to my site repository as git submodules. They were added using the SSH protocol (my workflow preference) rather than the HTTPS protocol. One theme is being used with my production branch (master) and the other with a staging branch with the intention that it will eventually be merged into master.

Deploys from either branch are not working. Below is the latest deploy log for the staging branch-deploy. The logs suggest I don’t have sufficient access rights but I naturally don’t have the access rights to the theme repositories since I’m not the author or a maintainer. Both are public repositories hosted on Github.

9:53:26 AM: build-image version: 0143b4617dd9b5b3faed2aefa29c54a846e69aae (focal)
9:53:26 AM: buildbot version: 0143b4617dd9b5b3faed2aefa29c54a846e69aae
9:53:26 AM: Building without cache
9:53:26 AM: Starting to prepare the repo for build
9:53:27 AM: No cached dependencies found. Cloning fresh repo
9:53:27 AM: git clone --filter=blob:none git@gitlab.com:doolio/ovolosash
9:53:27 AM: Preparing Git Reference refs/heads/staging
9:53:30 AM: Failed during stage "preparing repo": Error checking out submodules: Submodule "themes/dot-org-hugo-theme" (git@github.com:cncf/dot-org-hugo-theme.git) registered for path "themes/dot-org-hugo-theme"
Submodule "themes/hugo-creative-theme" (git@github.com:digitalcraftsman/hugo-creative-theme.git) registered for path "themes/hugo-creative-theme"
Cloning into "/opt/build/repo/themes/dot-org-hugo-theme"...
Warning: Permanently added "github.com" (ECDSA) to the list of known hosts.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of "git@github.com:cncf/dot-org-hugo-theme.git" into submodule path "/opt/build/repo/themes/dot-org-hugo-theme" failed
Failed to clone "themes/dot-org-hugo-theme". Retry scheduled
Cloning into "/opt/build/repo/themes/hugo-creative-theme"...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of "git@github.com:digitalcraftsman/hugo-creative-theme.git" into submodule path "/opt/build/repo/themes/hugo-creative-theme" failed
Failed to clone "themes/hugo-creative-theme". Retry scheduled
Cloning into "/opt/build/repo/themes/dot-org-hugo-theme"...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of "git@github.com:cncf/dot-org-hugo-theme.git" into submodule path "/opt/build/repo/themes/dot-org-hugo-theme" failed
Failed to clone "themes/dot-org-hugo-theme" a second time, aborting
: exit status 1: Submodule "themes/dot-org-hugo-theme" (git@github.com:cncf/dot-org-hugo-theme.git) registered for path "themes/dot-org-hugo-theme"
Submodule "themes/hugo-creative-theme" (git@github.com:digitalcraftsman/hugo-creative-theme.git) registered for path "themes/hugo-creative-theme"
Cloning into "/opt/build/repo/themes/dot-org-hugo-theme"...
Warning: Permanently added "github.com" (ECDSA) to the list of known hosts.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of "git@github.com:cncf/dot-org-hugo-theme.git" into submodule path "/opt/build/repo/themes/dot-org-hugo-theme" failed
Failed to clone "themes/dot-org-hugo-theme". Retry scheduled
Cloning into "/opt/build/repo/themes/hugo-creative-theme"...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of "git@github.com:digitalcraftsman/hugo-creative-theme.git" into submodule path "/opt/build/repo/themes/hugo-creative-theme" failed
Failed to clone "themes/hugo-creative-theme". Retry scheduled
Cloning into "/opt/build/repo/themes/dot-org-hugo-theme"...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of "git@github.com:cncf/dot-org-hugo-theme.git" into submodule path "/opt/build/repo/themes/dot-org-hugo-theme" failed
Failed to clone "themes/dot-org-hugo-theme" a second time, aborting
: exit status 1
9:53:30 AM: Error checking out submodules: Submodule "themes/dot-org-hugo-theme" (git@github.com:cncf/dot-org-hugo-theme.git) registered for path "themes/dot-org-hugo-theme"
Submodule "themes/hugo-creative-theme" (git@github.com:digitalcraftsman/hugo-creative-theme.git) registered for path "themes/hugo-creative-theme"
Cloning into "/opt/build/repo/themes/dot-org-hugo-theme"...
Warning: Permanently added "github.com" (ECDSA) to the list of known hosts.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of "git@github.com:cncf/dot-org-hugo-theme.git" into submodule path "/opt/build/repo/themes/dot-org-hugo-theme" failed
Failed to clone "themes/dot-org-hugo-theme". Retry scheduled
Cloning into "/opt/build/repo/themes/hugo-creative-theme"...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of "git@github.com:digitalcraftsman/hugo-creative-theme.git" into submodule path "/opt/build/repo/themes/hugo-creative-theme" failed
Failed to clone "themes/hugo-creative-theme". Retry scheduled
Cloning into "/opt/build/repo/themes/dot-org-hugo-theme"...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of "git@github.com:cncf/dot-org-hugo-theme.git" into submodule path "/opt/build/repo/themes/dot-org-hugo-theme" failed
Failed to clone "themes/dot-org-hugo-theme" a second time, aborting
: exit status 1
9:53:30 AM: Failing build: Failed to prepare repo
9:53:31 AM: Finished processing build request in 4.863s

My .gitmodules file at the root of my repository:

[submodule "themes/hugo-creative-theme"]
	path = themes/hugo-creative-theme
	url = git@github.com:digitalcraftsman/hugo-creative-theme.git
[submodule "themes/dot-org-hugo-theme"]
	path = themes/dot-org-hugo-theme
	url = git@github.com:cncf/dot-org-hugo-theme.git

The relevant sections from my .git/config file:

[submodule "themes/hugo-creative-theme"]
	url = git@github.com:digitalcraftsman/hugo-creative-theme.git
	active = true
[submodule "themes/dot-org-hugo-theme"]
	url = git@github.com:cncf/dot-org-hugo-theme.git
	active = true

My /themes/ and .git/modules/themes/ directories are populated with both themes as you would expect.

At first the issue seemed isolated to my branch-deploy but following the advise given by a netlify support engineer here on another (similar) topic I re-linked my site to my netlify account. The result of this was deploys now fail also on my production branch :frowning_face:. Thanks for your time.

2

There is an entire support guide on this specific topic that should help you clear things up:

3

Thanks but I actually read that guide before writing this post. I was already managing my theme as a submodule. The difference is that I’m using the SSH protocol rather than the HTTPS protocol. Your documentation here suggests I can use the SSH protocol. The guide you linked uses HTTPS. I guess the question is can I only use SSH if I own the theme repository? I don’t.

Now I have a deploy key linking my site repository to netlify. I’m assuming if I use SSH for my theme submodule I need a deploy key for its repository too but how can I do that if I don’t own that repository. Thanks.

4

Hi, @doolio. The solution here is to use HTTPS instead of SSH for any public repositories you want to clone.

Netlify doesn’t have your SSH key. Netlify is using the Netlify GitHub app to connect to your repo. Using SSH for the submodules means some SSH key for GitHub is required. Quoting the documentation you linked to previously:

If the repository is private, or if you prefer to use ssh format (for example, git@github.com:owner/project.git), you will need to follow the instructions below to generate a deploy key in Netlify and add it to the submodule repository settings.

So, if you really want to use SSH to clone the submodule you can add a deploy key to to site and GitHub account to allow this. However, this extra work is not required. If you change the submodule references to HTTPS they will be successfully cloned without configuring a deploy key.

5

Thanks for your input Luke. I will revert to HTTPS as you suggest and report back if I still have issues.

But just for my education. I have an SSH account key on both my Github and Gitlab accounts. I also have a deploy key (also an SSH key I presume) for my site respository hosted on Gitlab. It is this deploy key which I believe grants Netlify read access to my site repository on Gitlab. If I understand you above correctly I would need to add this same deploy key (as I believe I can only have one in my Netlify account) to the theme repository that is hosted on Github but I can’t do that since I don’t own that repository. Right?

6

Switching to HTTPS resulted in a successful deployment. One thing I noted though is that I see the (branch) deploy was performed twice in the list of my deploys.

1 Like
7

thanks for sharing this with the community.

8

Hi, @doolio. I wanted to follow-up to answer you earlier question. Most of what was stated before the question was correct above but there are some details I want to clarify. I’m going to take it sentence by sentence:

I have an SSH account key on both my Github and Gitlab accounts.

While I cannot verify this personally, I’m certain as I can be you are correct.

I also have a deploy key (also an SSH key I presume) for my site respository hosted on Gitlab. It is this deploy key which I believe grants Netlify read access to my site repository on Gitlab.

This is also correct, but please note that the submodule is on GitHub and not on GitLab:

	url = git@github.com:digitalcraftsman/hugo-creative-theme.git

Which takes us to this:

If I understand you above correctly I would need to add this same deploy key (as I believe I can only have one in my Netlify account) to the theme repository that is hosted on Github but I can’t do that since I don’t own that repository. Right?

No, you don’t need to add the GitHub SSH key to the submodule repo. The repo is public so that isn’t required. However, when cloning via SSH one must use some SSH key. Netlify doesn’t have your GitHub SSH key which is why we cannot clone via SSH the GitHub repo. For GitHub access, we use the Netlify GitHub app for access control to the repos. This is why you must either add the deploy key for GitHub to the site (because Netlify doesn’t have it yet) or use the HTTPS clone method.

You had most of the details correct and I just wanted to close the loop on the minor details that were not.

1 Like
9

That’s all much clearer now. Thanks Luke. I appreciate you taking the time to clarify each point. Hopefully, this thread can help others in future as well.