How to control a Netlify build and deploy via CI

Support
,
1

I’ve been working to try to take tighter control over the Netlify build and deploy process via my CI server (CircleCI). I’m finding it a bit tricky though to figure out how to break down the steps and make sure that they’re happening correctly.

I’d like to have manual control over the different steps but it feels like the API doesn’t give scope for what I need. I’ve switched off automatic builds.

Ideally the setps I’d like to be able to execute via the API

  1. Trigger Netlify to fetch the latest version of the code from Github. I’d ideally like to specify a commit hash so there are no race condition errors. I want to do this manually during release and not using a Github callback.
  2. To build the branch and return the ID of the build but then to hold from publishing it.
  3. At a future point once other CI tests have passed, to release Netlify to publish the build to production.

Doing the above has proved to be harder than I’d hoped though.

**The problems I’m having **

Step 1 - fetch code from Github for particular commit
I can’t see any way via the API to tell Netlify to “get the latest code from Github ready to build”. It seems that Netlify either works automatically off github hooks or it doesn’t fetch from Github at all. Is there an in between where you can manually tell it to fetch the head and build it?

Without being able to manually trigger this step it’s not clear which the build that corresponds to the commit I’m releasing actually is.

Step 2 - build and hold
As far as i can tell, there’s no way of triggering the "fetch code from github as described above. Because of this you, this step needs to just make the assumptioin that the code has been fetched and built already. It can’t request it and process the outcome (or failure).

I need to just assume it’s been done. Then I need to figure out which build corresponds to the particular commit that I’m running throuhg CI.

To do that I query listSiteDeploys and interrogate it to see which is the most recent for the particular commit I want to deploy. This is all doable but it would be much nicer to be able to just request the deploy and then store the ID of that particular deploy rather than decoupling the two processes. Also what if it hasnt’ happened yet or there’s an error - the asyncrhonicity makes it hard to be sure what’s really happening.

Step 4 - publish the build to production

After having played with it a lot, there seem to be three (or four depending on your setup) steps in the netlify deploy process

  • build
  • deploy
  • (unlock if you’ve locked deploys)
  • publish.

At this point in the process we’re looking to unlock and publish. However the API doesn’t seem to give explicit access to publishing. Locking deploys prevents them from automatically publishing them but unlocking doesn’t seem to publish them. A workaround for this though seems to be skipping the unlock altogether and going straight to restoreSiteDeploy. Some of this all feels a bit smudgy to me :face_with_monocle:.

If you’re still with me then thank you for persevering. In summary what I’d like to know is this:

  1. Is there a way via the CLI or API to ask Netlify to “git pull this particular commit” and then to “build but not deploy the commit” and then to return the ID of that build?
  2. Is there an explicit API endpoing for unlocking deploys or should I just continue using the restoreSiteDeploy one?
  3. I’d prefer to use Netlify to do the build work rather than building it on my CI and then sending the files over but would that be the only other best option? It seems much more laborious than having Netlify do it.

Thank you

2

Hey @peternixey,
As @fool says here:

The most important advice I can give you about using our API is to watch how we do it! Our entire admin UI works via this API, so checking to see how we use the API in-app will be the best guide to using every feature.

So that’s one thing! Another thing is that the open-api site is a great overview, but does not document everything and is not as well maintained as, well, our UI.

Now into your TL;DR:

  1. Is there a way via the CLI or API to ask Netlify to “git pull this particular commit” and then to “build but not deploy the commit” and then to return the ID of that build?

I’m… not sure what you’re asking here. Maybe @luke will know?

  1. Is there an explicit API endpoing for unlocking deploys or should I just continue using the restoreSiteDeploy one?

The UI is your friend on this one! I think a POST to https://api.netlify.com/api/v1/deploys/DEPLOY_ID/unlock will unlock, but then you will have you restore. I believe we’ve decoupled these because people who “unlock” in the UI don’t necessarily want to publish whatever deploy they’ve unlocked.

  1. I’d prefer to use Netlify to do the build work rather than building it on my CI and then sending the files over but would that be the only other best option? It seems much more laborious than having Netlify do it.

We’re biased, but we think you should Netlify do the build work :slight_smile: Our whole product is meant to make building and deploying smooth and straightforward. There are obviously assumptions baked in that might not align with what you want to build, but we think they’re good assumptions that make people’s lives easier. If there are improvements you want, let us know here, see what you can accomplish with build plugins, or check out some of other repos that handle different parts of the build process.

Hope this is somewhat helpful- let us know if you have more questions!