[Support Guide] Linking a repository via API

Hey please let me know how that goes for you, looks like I might have to go that route as well. my discord ihorbond#5688 cheers

Hey there folks, thanks for your patience! I have shared this with my team to investigate further. We will follow up on this thread!

1 Like

My guess is that something in our API changed slightly vs the code examples I shared. If you’d like to try to reverse engineer things - you need to do essentially what our UI does, and this article describes how to discover the new call(s) you’ll need, in case something has changed: [Support Guide] Understanding and using Netlify's API

If you aren’t interested in that, we’d need to know more details to actually help you, which we’ll be happy to do :slight_smile: What does “doesn’t work” look like in more detail, please? Happy to help you get things sorted, but you’ll want to follow our guide on asking good questions to help us understand what you are seeing: Asking great questions to get great answers

Thanks. I’m trying to reverse engineer but finding a different set of fields between the UI and the documentation link you provided. For instance, in UI the call is
POST /api/v1/ihorbond/sites with the same set of fields under “repo” field as shown here Get started with the Netlify API | Netlify Docs and essentially in this support post.
I’m using the same set for my API call.
This documentation on the other hand shows a different set of fields
Netlify API documentation
I will now try the above but would be good to have some clarity on which one is actually the preferred/correct way.
Thanks for your help.

UPDATE: Woohoo looks like the Netlify API documentation works!

Well this is super weird. Now that I was able to successfully create it even sending a request like this works fine i.e CI/CD is getting setup properly

{
    "name":"",
    "repo": {
        "provider":"github",
        "repo":"zerocodenft/nuxt-nft-dapp-template",
        "private":"false",
        "branch":"master",
        "cmd":"npm run generate",
        "dir":"dist"
    }
}

Mind you that git repository doesn’t even have a deployment key setup. Is the deployment key and installation ID cached on netlify side so that subsequent requests just magically work or something ?

UPDATE: created brand new Netlify account. Confirmed: can deploy from public repo without deploy_key or installiation_id. I guess this is common sense thing since repo is public? Might still be worth mentioning in the docs

I’d be shocked if you - that is, Netlify on your behalf, could pull from that repo - can you/we?

There is no default value for deploy_key_id since we generate one deploy key per site (well, we used to; now sites linked through our UI use the github app instead, if your repo[installation_id] is set correctly so maybe that’s what you are seeing?)

How can I pull from that repo with Netlify on my behalf ?
That makes sense. If Netlify linked the repo to the installation_id of the Netlify App on Github account that hosts that public repository it appears that all subsequent calls to POST api/v1/sites will just work and setup CI/CD automatically. This is not ideal in case the public repo owner makes it private. The CI/CD on this unrelated account will keep working and getting updates (I tested). I can also see the installation_id if I call GET /api/v1/sites using this brand new Netlify account I’ve created and its access token.

I have 2 more questions please:

  1. After successful POST /api/v1/sites I only get back build_id and deploy_id is empty. Is there a way to get it somehow? I did see that if I GET /api/v1/builds/:id it returns deploy_id. I’m just looking for the most efficient way to poll the status of the site so I can present to the end user.
  2. Initially after creating the site I see 2 builds in the queue right away which seems unnecessary. Any way to reduce it to just 1 ? (I had to manually cancel the first one, it got stuck)

Looks like creating a github hook doesn’t work anymore either.

Github is successfully sending the webhook event, and getting a 204 response, but netlify doesn’t kick off the build :frowning:

It seems like this is broken for private repos

Hey folks and sorry to be slow to follow up:

@ihorbond:

  1. in my last message, I was trying to say that “I’d be shocked if you had successfully configured things for our build system to be able to pull from your repo to build your code”, which I believe is what you were trying to configure?

  2. I think that you are polling in the best way possible. I won’t again share with you the doc about reverse engineering our API but that is still where you should look at how our UI is doing the same poll (which it does every minute that you have the deploy listings page open :))

  3. re: two builds, I believe we automatically create one for you when you link the site. So you shouldn’t need to make an “initial commit” if you already have the code in your repo when you link it.

To both @ihorbond and @kgwebsites could you clarify if what you are trying to link is a private repo within a GitHub organization or not? If so, you’d need to be on the pro plan to link it in our UI or via API, per our pricing page:

It is mentioned under the pro plan as:

  • Automated builds from private organization Git repos

(private repos not in organizations should not have this limitation).

If you’re having trouble getting this to work on a repo like that on a Pro team, let me know - I could only see a Starter team for you in our database.

2 Likes

Hi,

I’m trying to link a private Gitlab repository to Netlify with Netlify API (with curl).

First I create deploy key with endpoint /deploy_keys.

Then I add Netlify webhook to my Gitlab project with these params :

url=https:%2F%2Fapi.netlify.com%2Fhooks%2Fgitlab&confidential_issues_events=false&confidential_note_events=false&deployment_events=false&enable_ssl_verification=true&issues_events=false&job_events=false&merge_request_events=true&note_events=false&pipeline_events=false&push_events=true&releases_events=false&tag_push_events=false&wiki_page_events=false

→ webhook is added and if I test a push event from gitlab i’m getting a 204, so it seems to work
→ I tried before with token param set to Netlify deploy public key but webhook was not added

Then, I update Netlify site with these params:

 {"build_settings":{"env":{"NODE_VERSION":"14"}},"repo":{"provider":"gitlab","id":XXXXXXXX,"repo_path":"my-private-repo-path","public_repo":false,"private_logs":true,"repo_branch":"prod","cmd":"npm\u0020run\u0020generate","dir":"dist","deploy_key_id":"XXXXXXXXXXXXX"}}

→ it seems to work but when I go to deploys page I can see deploy failed with these details :

6:49:44 PM: Build ready to start
6:49:46 PM: build-image version: 122b31996ccaffd45d820a452d6227f8312110cc (focal)
6:49:46 PM: build-image tag: v4.5.3
6:49:46 PM: buildbot version: 94cf4881e10a7b2d529971a176d6ede22bcaeb38
6:49:46 PM: Fetching cached dependencies
6:49:46 PM: Failed to fetch cache, continuing with build
6:49:46 PM: Starting to prepare the repo for build
6:49:47 PM: User git error while checking for ref refs/heads/prod
6:49:47 PM: Creating deploy upload records
6:49:47 PM: Failing build: Failed to prepare repo
6:49:47 PM: Failed during stage 'preparing repo': error checking for ref: Warning: Permanently added 'gitlab.com' (ECDSA) to the list of known hosts.

git@gitlab.com: Permission denied (publickey,keyboard-interactive).

fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
: exit status 128
6:49:47 PM: Finished processing build request in 914.953513ms

With another Netlify site, I linked another private repository manually using Neltify UI and it works fine, when I compare both configurations on Netlify and Gitlab all seems to be equal.

Could you tell me what I’m doing wrong please?

@fool you said on your first post “you’ll need to put the public_key in place at your git provider”, but where do I have to do that? In my Gitlab profile in ssh keys? I didn’t do that with the Netlify/Gitlab link I added manually and it works…

Thanks for your help!

Finally I had added the deploy key in project settings and not in profile settings ! I didn’t remember sorry…
It works fine now!

1 Like

Thanks for coming back and letting us know everything is working for you, @m3ways. Happy building :netliconfetti:

Hi, it works for the site creation and the first build/deploy but then when I push another commit, nothing happens. I have created a deploy_key_id, added the public key on gitlab.

1 Like

in the new commit you are pushing, are there changes that we would be picking up?

if you aren;t sure, can you make a small change (such as some comments etc) that will force us to recheck the whole file? i am wondering if we are not picking up the necessary changes if there is no file change.

yes I have added a comment and nothings happens.
here is how i create a new site:

{
    "name": "cms-generator-newsite",
    "repo": {
        "cmd": "npm run build",
        "dir": ".next",
        "allowed_branches": [
            "master"
        ],
        "provider": "gitlab",
        "repo_type": "git",
        "repo_url": "https://gitlab.com/<path>",
        "repo_branch": "master",
        "repo_path": <repo path>,
        "deploy_key_id": <deploy key>
    }
}

When it is created, it builds and deploys well but then nothing happens if I push a new commit.

Hi, @brunocc. You must also create a webhook for the site at Netlify and then add that webhook to GitLab so their service will trigger builds at Netlify when a push to the upstream repo occurs.

Our API cannot do this at GitLab. The GitLab APIs are used for this.

When you add a site using GitLab via the web UI we create the webhook using our API. We use your GitLab credential to make API calls at GitLab to add the new webhook to their service. This is how GitLab tells Netlify when a new commit occurs.

So, there are two extra steps:

  1. Make an API call at Netlify to make a new webhook for the site.
  2. Make an API call at GitLab to add that webhook to the repo.

If you do this in our web UI and open the browser devtools you will be able to see the exact API calls made to do this and what JSON content the API calls required in the Network tab.

When those steps are completed commits to the GitLab repo will trigger builds at Netlify. If there are questions about this, please let us know.

@fool it might be worth updating this post to explain that the repo id for bitbucket accounts consists of workspace/repoName rather than an integer id

great suggestion! Just updated that:

Note that there are two ID’s there - a GitHub repo ID (which you can get from their API ) - and similar ID’s are needed for GitLab and BitBucket as well - though for bitbucket at least the ID will have the format login/project instead of being a literal ID number.

Think that makes it clear enough? Do you think I need to call it out further anywhere else? You are my target audience so happy to take your suggestion, @shallow_alchemy :slight_smile: