Starting a Gitlab pipeline after a successful Netlify deployment

Support
, ,
#1

Hi community,

I’m using a self-hosted Gitlab repo and successfully connected it to Netlify for website deployments (on master and other branches).

What I’m now looking to do is to perform E2E testing on the deployed site, as part of a GitLabCI pipeline. I’ve read the docs on Outgoing Webhooks for Netlify, and GitLab’s webhook docs. Basically, I need to pass the site’s Netlify URL to GitLab via the Webhook, but GitLab expects POST data specified via FORM variables, not as a JSON object (which is what Netlify sends).

Any ideas on how I can get Netlify to start an E2E Gitlab pipeline?

#2

Hiya, welcome!

For setting up tests to run before deployment that then trigger a build, would something like this work for you? https://dev.to/sidhantpanda/self-hosted-gitlab-continuous-deployment-to-netlify-49lk

If you go that route, you’ll need to configure your Netlify URL and API ID in the GitLab UI. You can find your site’s API ID here:
https://app.netlify.com/sites/YOURSITE/settings/general#site-information

Our docs have more details about notifications going the other way, from Netlify to GitLab:

I’m not sure if this answers your question, so feel free to let us know if we can help further!

#3

Hi Jen,

Yes, I could run E2E tests before deploying, but would that really be an E2E test?

Call me old fashioned, but I would expect a build-deploy pipeline to consist of a sequence like this: unit tests, component tests, build, deploy, E2E tests, performance tests.

If Netlify can do the [unit tests, component tests, build, deploy] steps (via the build command), that saves me doing it via my own pipeline. But if there is no way to do [E2E, performance] by way of Netlify notifiying GitLab after a successful deployment, then I guess I’ll have to figure out another way to do this (maybe an AWS Lambda, or Netlify Cloud function to convert the Netlify-JSON to Gitlab form data).

Thanks for responding so quickly.

#4

chiming in from the sidelines:

has some information & ideas on e2e tests. Not sure if you saw that already?

#5

Thanks Perry - that was a good background article on testing with Netlify, that I had not seen. I still need my own Netlify-JSON-to-gitlab-Form-data conversion Lambda though, it seems, if I want to do system integration testing.

#6

depends - you could probably use the webhook to trigger tests without any parsing. You’d just need your test harness to make some API calls to Netlify’s API to find the latest build and its URL. But GitLab knows that already, so maybe you can just leverage its internal logic? Not sure, haven’t used GitLab’s pipelines before.

#7

Thanks “fool”.

FWIW, I implemented a Netlify cloud function for the deploy-succeeded trigger, which parses the Netlify event into a format that can trigger a Gitlab pipeline. It requires node-fetch and two environment vars to be setup:

/**
 * Netlify-GitLab Webhook transformer
 *
 * This lambda function converts the Netlify JSON event into a GitLab webhook request (which expects form-data for the body).
 * It is triggered on a successful deploy in Netlify.
 *
 * The function name MUST be "deploy-succeeded/deploy-succeeded.js" for everything to work correctly.
 */

const fetch = require('node-fetch');
const { URLSearchParams } = require('url');

exports.handler = async function(event) {
  console.log('Deploy succeeded');

  const gitlabWebhook = process.env.GITLAB_DEPLOY_NOTIFICATION_URL || '';
  const gitLabToken = process.env.GITLAB_TOKEN || '';

  const eventBody = JSON.parse(event.body);
  const input = eventBody.payload;

  const params = new URLSearchParams();
  params.append('token', gitLabToken);
  params.append('ref', input.branch);
  params.append('variables[SITE_URL_FROM_NETLIFY]', input.url);

  try {
    const response = await fetch(gitlabWebhook, { method: 'POST', body: params });
    if (!response.ok || response.status >= 400) {
      console.log('Failure');
      // NOT res.status >= 200 && res.status < 300
      return { statusCode: response.status, body: response.statusText }
    }
    console.log('result', response);

    const data = await response.json();
    console.log('data', data);

    return {
      statusCode: 200,
      body: JSON.stringify(data)
    };
  } catch (err) {
    return { statusCode: 400, body: err.message || JSON.stringify(err) }
  }
};

Environment variables:
GITLAB_DEPLOY_NOTIFICATION_URL = https://your.gitlab.site/api/v4/projects/:project-id/trigger/pipeline
GITLAB_TOKEN = See https://docs.gitlab.com/ee/ci/triggers/#adding-a-new-trigger

4 Likes
#8

Thanks for sharing your solution with the community, @uglow. Much appreciated!