Deploys: yarn version & pushing to git

Apologies, as maybe I’m coming at this problem from the wrong angle here: I have a yarn netlify task in my package.json that netlify is running to spin out the build of my site.

The script is very simple, it’s: yarn version --patch && git push && git push --tags && yarn build, however netlify keeps cancelling the build because it complains Author identity unknown during the git push stage(s). It advises to set the git commit author but not sure I can / want to do that in the pipeline. I had a look but couldn’t see anything related to setting a git user during the deployments.

Any advice? I was hoping to have the version automatically bumped during netlify deployments, but may be impossible from the angle I’m taking. I’ve used other tools like Jenkins that lets you set a git user to use during builds - so maybe it’s just a configuration issue?

Hi and welcome to the community @amaccann

This will not be an official answer from Netlify, but I’ve played a little with “committing data to a repo at the time of a Netlify build.” I still need to do a writeup on this, but here is my take.

The build path is not really a checkout of your repo on Netlify like you would think it is. It’s more of an environment built from the checking out of your repo. You don’t really want to use this as your source of truth for your repository. Knowing this, we can still accomplish what you want, but have to accomplish it with a “working” environment path that will do a full checkout of your repository. Basically, build a development checkout of our repository, run the commands we want and check it back into our repo.

You could use a bash shell script to run the commands in serial and do what you want using your checked out repository. The following bash script backs up a snapshot of the data from the current Netlify build git repository after the data is pulled from another source. Not quite what you are looking for, but gives you an example of how I set it up.

/bin/gitpush.sh

#!/bin/bash
echo "Starting the git backup"

git clone https://$GIT_USERNAME:$GIT_TOKEN@$GIT_REPO_URL $GIT_REPO_NAME
cd $GIT_REPO_NAME
ls -l
git remote rm origin
git remote add origin https://$GIT_USERNAME:$GIT_TOKEN@$GIT_REPO_URL
git remote -v
git config --local user.email $GIT_EMAIL
git config --local user.name $GIT_USERNAME
git checkout main
git pull origin main
cp -a ../site/data/. site/data/
git status
git add .
git commit -m "commit by :robot: [skip ci]"
git push --set-upstream origin main

echo "Ending the git backup"

The environment variables are stored in the Netlify admin and this is a private repo. I believe all of these command tokens are obfuscated, but you would want to make sure prior to releasing this as a public repo.

I’m not sure they would be obfuscated. I wrote a node script equivalent that did that for me, now that I reviewed it. The following is the equivalent of the above shell script using a node script instead that obfuscates the output of the token :tada:

/scripts/git-push.js

const fs = require("fs");
const path = require("path");
const { spawnSync } = require("child_process");

const {
  GIT_TOKEN,
  GIT_USERNAME,
  GIT_EMAIL,
  GIT_REPO_URL,
  GIT_REPO_NAME
} = process.env;
// leaving this without https:// in order to reuse it when adding the remote
// const GIT_REPO_URL = "github.com/org/this-repo-name.git";
// const GIT_REPO_NAME = "this-repo-name"; // or an optional name

const cleantoken = new RegExp(`${GIT_TOKEN}`, 'g');
const clean = str => str.replace(cleantoken, '[private key]')

function runCommand(commandString) {
  const options = {
    stdio: 'pipe',
    encoding: 'utf-8'
  }
  const [command, ...args] = commandString.match(/(".*?")|(\S+)/g);
  const cmd = spawnSync(command, args, options);
  console.log(
    `$ ${clean(commandString)}
${clean(cmd.stdout)}`
  )

  // Error on status
  if (cmd.status !== null && cmd.status !== 0){
    console.log(`${clean(cmd.stderr)}`)
    throw new Error(`Git backup failed`);
  }
}

// clone the repository and set it as the cwd
// sadly, clone doesn't support --porcelain
const repoPath = path.join(process.cwd(), GIT_REPO_NAME)
if (fs.existsSync(repoPath)) {
  // already checked out and cached into build
  console.log(`Already exists: ${repoPath}`)
} else {
  console.log(`Cloning into ${GIT_REPO_NAME}`)
  runCommand(`git clone --quiet https://${GIT_USERNAME}:${GIT_TOKEN}@${GIT_REPO_URL} ${GIT_REPO_NAME}`)
}
process.chdir(repoPath)

runCommand(`ls -l`)
// setup the origin for the repository with personal access token
runCommand(`git remote rm origin`)
runCommand(`git remote add origin https://${GIT_USERNAME}:${GIT_TOKEN}@${GIT_REPO_URL}`)
runCommand(`git remote -v`)
// update local git config with email and username (required)
runCommand(`git config --local user.email ${GIT_EMAIL}`);
runCommand(`git config --local user.name ${GIT_USERNAME}`);
// setup the branch to HEAD
runCommand(`git checkout master`)
runCommand(`git pull origin master`)
// copy data command(s) here
runCommand(`cp -a ../site/data/. site/data/`)
// stage local files
runCommand(`git status --porcelain`)
runCommand(`git add .`);
// commit changes
const roboMessage = `commit by :robot: [skip ci]`
runCommand(`git commit -m "${roboMessage}"`);
// push to branch
runCommand(`git push --porcelain --set-upstream origin master`);

I’d be interested in Netlify’s take on what I’m doing here to see if this is recommended or there is an easy answer to your question. Sorry for all the edits. Been a long time since I worked on this solution.

-T

Hey folks! @talves asked us to sound off as to whether his approach was problematic and the answer is “not for Netlify, but maybe for you?” This article around “how to access repositories during build” shows some patterns similar to Tony’s:

That is talking about reading, but the permissions you grant at your git provider will determine if you can push or just pull; the config is identical here.

I do think you’d want to be EXTREMELY careful about committing anything giving you access to your git repo, TO your git repo - one leak of source code and people can add backdoors to your code. I and Netlify always try to keep secrets out of code. One place you can add them is in our environment variables as Tony mentions; the ones in our UI are safe from anyone who doens’t have access to your team’s admin pages.

Since I think Tony may be writing this up in more detail, also wanted to point out a different way to approach the same problem - pushing during build - I do it using SSH keys directly and demonstrate how, here:

I don’t think the GH “native” auth tokens are so long or oddly formatted that you’d need to go to that level, so it is probably smarter to use them instead, but I always like to present a few solutions if they are available and let you survey and choose…