[Support Guide] Linking a repository via API

Last reviewed by Netlify Support Staff: August 2023

It is easiest to link a repository in Netlify’s UI; doing it via API is challenging and should be a rare use case. Nonetheless, it is possible to create a Netlify site linked to a repository at any of our supported git providers programatically. Specifically, it is possible to use Netlify’s REST API for these situations. You’ll probably be interested in the general topics around “how do I use Netlify’s REST API?” (see this support guide, as well as our public documentation on the topic).

You can link a repository while you create a site. This is an example (partial) payload from such a successful site creation (sent via HTTP POST to https://api.netlify.com/api/v1/sites).

{
  "repo": {
    "branch": "main",
    "cmd": "jekyll build",
    "deploy_key_id": "5a908857a6188f466bed"
    "dir": "_site/",
    "private": false,
    "provider": "github",
    "repo": "account_name/repository_name",
    "repo_id": 74777123
  }
}

Note that there are two ID’s there - a GitHub repo ID (which you can get from their API) - and similar ID’s are needed for GitLab, BitBucket, and Azure DevOps - though, for bitbucket (at least), the ID will have the format login/project or workspaceName/project instead of being a literal ID number. The second ID is a deploy key ID . You’ll need to create the Deploy Key via our API before you’ll be able to create a usable repo object, so that is the call you have to make FIRST, before creating the site.

The call to create a deploy key is an HTTP POST to https://api.netlify.com/api/v1/deploy_keys . This call responds with JSON containing public_key and id parameters (you’ll need to put the public_key in place at your git provider. For most of the other providers this can be done on the repo level i.e adding it to repository settings but for Azure DevOps add this to user settings under SSH keys. Due to this limitation in Azure, a good approach is using a service account in case a user’s access is revoked), and keep the id value, for using with the repo definition I already quoted.

You’ll also need to change the value of provider based on what you use - “github”, “gitlab”, “bitbucket”, “azure_devops”, and “manual” are the options. manual is intended for anything other than repos hosted at any of github.com, gitlab.com, bitbucket.org, or dev.azure.com as of June 2020. In July 2020, we created the ability for Business & Enterprise customers to link their self-hosted (GitHub Enterprise/GitLab self-hosted) repos to our service directly, so please ping us in the helpdesk (all non-Starter users can email support@netlify.com directly from their Netlify login email address) if you’re at that account level and need API help setting up a self-hosted repo. Note that, as on August 2023, the Business plan has been deprecated, so the self-hosted repo feature should now be restricted to the Enterprise plan.

Finally, please be aware that you need to configure your git provider to build in this case. When you link a site through our UI, we programatically configure the settings required at your git provider to tell us about your commits and merges. When you instead do it yourself - you also need to configure that part yourself! You can either:

  1. create a webhook to https://api.netlify.com/hooks/github (or gitlab, or bitbucket, or azure_devops as that last path component). This webhook should be:
    • using content-type application/json
    • POST on the events you want us to know about: likely commit, and potentially PR/merge/branch deletion events.

OR

  1. if and only if you are on GitHub: set up a sample site through our UI, linking with the GitHub app (this is the only way to link a GitHub repo through our UI for the past 2.5 years or so), find the “installation ID” of our GitHub App (it will be the final path component when you click “configure” on our installed app at https://github.com/orgs/YOURORGNAME/settings/applications or https://github.com/ACCOUNTNAME/settings/applications - it will be an integer), and include that in the repo object as quoted above, as a parameter like this:

"installation_id": "X",

Either of these workflows will tell the git provider to tell us about commits.

You can also use our CLI to link a repo, see this forum thread for more details.

If you have further questions, please start a new thread with your site name, a link to your repo, and the steps you have tried. Thanks!

6 Likes

Does the installation_id method still work ? The “https://github.com/orgs/YOURORGNAME/settings/applications” link doesn’t work anymore I believe it’s now “Sign in to GitHub · GitHub” so I copied the ID and passing it in along with other data. Site gets created but CI/CD is not setup. Please advise.
I followed these steps:

  1. Get deploy key from Netlify,
  2. Create github repo from template
  3. Assign deploy key from netlify to git repo
  4. Deploy Netlify site with installation_id and deploy_key_id.
    No errors but site is not linked. Checking on git repo it says key was never used
1 Like

+1 it doesn’t look like this method is working anymore. Gonna have to try the webhook for now.

Hey please let me know how that goes for you, looks like I might have to go that route as well. my discord ihorbond#5688 cheers

Hey there folks, thanks for your patience! I have shared this with my team to investigate further. We will follow up on this thread!

1 Like

My guess is that something in our API changed slightly vs the code examples I shared. If you’d like to try to reverse engineer things - you need to do essentially what our UI does, and this article describes how to discover the new call(s) you’ll need, in case something has changed: [Support Guide] Understanding and using Netlify's API

If you aren’t interested in that, we’d need to know more details to actually help you, which we’ll be happy to do :slight_smile: What does “doesn’t work” look like in more detail, please? Happy to help you get things sorted, but you’ll want to follow our guide on asking good questions to help us understand what you are seeing: Asking great questions to get great answers

Thanks. I’m trying to reverse engineer but finding a different set of fields between the UI and the documentation link you provided. For instance, in UI the call is
POST /api/v1/ihorbond/sites with the same set of fields under “repo” field as shown here Get started with the Netlify API | Netlify Docs and essentially in this support post.
I’m using the same set for my API call.
This documentation on the other hand shows a different set of fields
Netlify API documentation
I will now try the above but would be good to have some clarity on which one is actually the preferred/correct way.
Thanks for your help.

UPDATE: Woohoo looks like the Netlify API documentation works!

Well this is super weird. Now that I was able to successfully create it even sending a request like this works fine i.e CI/CD is getting setup properly

{
    "name":"",
    "repo": {
        "provider":"github",
        "repo":"zerocodenft/nuxt-nft-dapp-template",
        "private":"false",
        "branch":"master",
        "cmd":"npm run generate",
        "dir":"dist"
    }
}

Mind you that git repository doesn’t even have a deployment key setup. Is the deployment key and installation ID cached on netlify side so that subsequent requests just magically work or something ?

UPDATE: created brand new Netlify account. Confirmed: can deploy from public repo without deploy_key or installiation_id. I guess this is common sense thing since repo is public? Might still be worth mentioning in the docs

I’d be shocked if you - that is, Netlify on your behalf, could pull from that repo - can you/we?

There is no default value for deploy_key_id since we generate one deploy key per site (well, we used to; now sites linked through our UI use the github app instead, if your repo[installation_id] is set correctly so maybe that’s what you are seeing?)

How can I pull from that repo with Netlify on my behalf ?
That makes sense. If Netlify linked the repo to the installation_id of the Netlify App on Github account that hosts that public repository it appears that all subsequent calls to POST api/v1/sites will just work and setup CI/CD automatically. This is not ideal in case the public repo owner makes it private. The CI/CD on this unrelated account will keep working and getting updates (I tested). I can also see the installation_id if I call GET /api/v1/sites using this brand new Netlify account I’ve created and its access token.

I have 2 more questions please:

  1. After successful POST /api/v1/sites I only get back build_id and deploy_id is empty. Is there a way to get it somehow? I did see that if I GET /api/v1/builds/:id it returns deploy_id. I’m just looking for the most efficient way to poll the status of the site so I can present to the end user.
  2. Initially after creating the site I see 2 builds in the queue right away which seems unnecessary. Any way to reduce it to just 1 ? (I had to manually cancel the first one, it got stuck)

Looks like creating a github hook doesn’t work anymore either.

Github is successfully sending the webhook event, and getting a 204 response, but netlify doesn’t kick off the build :frowning:

It seems like this is broken for private repos

Hey folks and sorry to be slow to follow up:

@ihorbond:

  1. in my last message, I was trying to say that “I’d be shocked if you had successfully configured things for our build system to be able to pull from your repo to build your code”, which I believe is what you were trying to configure?

  2. I think that you are polling in the best way possible. I won’t again share with you the doc about reverse engineering our API but that is still where you should look at how our UI is doing the same poll (which it does every minute that you have the deploy listings page open :))

  3. re: two builds, I believe we automatically create one for you when you link the site. So you shouldn’t need to make an “initial commit” if you already have the code in your repo when you link it.

To both @ihorbond and @kgwebsites could you clarify if what you are trying to link is a private repo within a GitHub organization or not? If so, you’d need to be on the pro plan to link it in our UI or via API, per our pricing page:

It is mentioned under the pro plan as:

  • Automated builds from private organization Git repos

(private repos not in organizations should not have this limitation).

If you’re having trouble getting this to work on a repo like that on a Pro team, let me know - I could only see a Starter team for you in our database.

2 Likes

Hi,

I’m trying to link a private Gitlab repository to Netlify with Netlify API (with curl).

First I create deploy key with endpoint /deploy_keys.

Then I add Netlify webhook to my Gitlab project with these params :

url=https:%2F%2Fapi.netlify.com%2Fhooks%2Fgitlab&confidential_issues_events=false&confidential_note_events=false&deployment_events=false&enable_ssl_verification=true&issues_events=false&job_events=false&merge_request_events=true&note_events=false&pipeline_events=false&push_events=true&releases_events=false&tag_push_events=false&wiki_page_events=false

→ webhook is added and if I test a push event from gitlab i’m getting a 204, so it seems to work
→ I tried before with token param set to Netlify deploy public key but webhook was not added

Then, I update Netlify site with these params:

 {"build_settings":{"env":{"NODE_VERSION":"14"}},"repo":{"provider":"gitlab","id":XXXXXXXX,"repo_path":"my-private-repo-path","public_repo":false,"private_logs":true,"repo_branch":"prod","cmd":"npm\u0020run\u0020generate","dir":"dist","deploy_key_id":"XXXXXXXXXXXXX"}}

→ it seems to work but when I go to deploys page I can see deploy failed with these details :

6:49:44 PM: Build ready to start
6:49:46 PM: build-image version: 122b31996ccaffd45d820a452d6227f8312110cc (focal)
6:49:46 PM: build-image tag: v4.5.3
6:49:46 PM: buildbot version: 94cf4881e10a7b2d529971a176d6ede22bcaeb38
6:49:46 PM: Fetching cached dependencies
6:49:46 PM: Failed to fetch cache, continuing with build
6:49:46 PM: Starting to prepare the repo for build
6:49:47 PM: User git error while checking for ref refs/heads/prod
6:49:47 PM: Creating deploy upload records
6:49:47 PM: Failing build: Failed to prepare repo
6:49:47 PM: Failed during stage 'preparing repo': error checking for ref: Warning: Permanently added 'gitlab.com' (ECDSA) to the list of known hosts.

git@gitlab.com: Permission denied (publickey,keyboard-interactive).

fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
: exit status 128
6:49:47 PM: Finished processing build request in 914.953513ms

With another Netlify site, I linked another private repository manually using Neltify UI and it works fine, when I compare both configurations on Netlify and Gitlab all seems to be equal.

Could you tell me what I’m doing wrong please?

@fool you said on your first post “you’ll need to put the public_key in place at your git provider”, but where do I have to do that? In my Gitlab profile in ssh keys? I didn’t do that with the Netlify/Gitlab link I added manually and it works…

Thanks for your help!

Finally I had added the deploy key in project settings and not in profile settings ! I didn’t remember sorry…
It works fine now!

1 Like

Thanks for coming back and letting us know everything is working for you, @m3ways. Happy building :netliconfetti:

Hi, it works for the site creation and the first build/deploy but then when I push another commit, nothing happens. I have created a deploy_key_id, added the public key on gitlab.

1 Like

in the new commit you are pushing, are there changes that we would be picking up?

if you aren;t sure, can you make a small change (such as some comments etc) that will force us to recheck the whole file? i am wondering if we are not picking up the necessary changes if there is no file change.

yes I have added a comment and nothings happens.
here is how i create a new site:

{
    "name": "cms-generator-newsite",
    "repo": {
        "cmd": "npm run build",
        "dir": ".next",
        "allowed_branches": [
            "master"
        ],
        "provider": "gitlab",
        "repo_type": "git",
        "repo_url": "https://gitlab.com/<path>",
        "repo_branch": "master",
        "repo_path": <repo path>,
        "deploy_key_id": <deploy key>
    }
}

When it is created, it builds and deploys well but then nothing happens if I push a new commit.