Hi and welcome to the community @amaccann
This will not be an official answer from Netlify, but I’ve played a little with “committing data to a repo at the time of a Netlify build.” I still need to do a writeup on this, but here is my take.
The build path is not really a checkout of your repo on Netlify like you would think it is. It’s more of an environment built from the checking out of your repo. You don’t really want to use this as your source of truth for your repository. Knowing this, we can still accomplish what you want, but have to accomplish it with a “working” environment path that will do a full checkout of your repository. Basically, build a development checkout of our repository, run the commands we want and check it back into our repo.
You could use a bash shell script to run the commands in serial and do what you want using your checked out repository. The following bash script backs up a snapshot of the data from the current Netlify build git repository after the data is pulled from another source. Not quite what you are looking for, but gives you an example of how I set it up.
/bin/gitpush.sh
#!/bin/bash
echo "Starting the git backup"
git clone https://$GIT_USERNAME:$GIT_TOKEN@$GIT_REPO_URL $GIT_REPO_NAME
cd $GIT_REPO_NAME
ls -l
git remote rm origin
git remote add origin https://$GIT_USERNAME:$GIT_TOKEN@$GIT_REPO_URL
git remote -v
git config --local user.email $GIT_EMAIL
git config --local user.name $GIT_USERNAME
git checkout main
git pull origin main
cp -a ../site/data/. site/data/
git status
git add .
git commit -m "commit by :robot: [skip ci]"
git push --set-upstream origin main
echo "Ending the git backup"
The environment
variables are stored in the Netlify admin and this is a private repo. I believe all of these command tokens are obfuscated, but you would want to make sure prior to releasing this as a public repo.
I’m not sure they would be obfuscated. I wrote a node script equivalent that did that for me, now that I reviewed it. The following is the equivalent of the above shell script using a node script instead that obfuscates the output of the token
/scripts/git-push.js
const fs = require("fs");
const path = require("path");
const { spawnSync } = require("child_process");
const {
GIT_TOKEN,
GIT_USERNAME,
GIT_EMAIL,
GIT_REPO_URL,
GIT_REPO_NAME
} = process.env;
// leaving this without https:// in order to reuse it when adding the remote
// const GIT_REPO_URL = "github.com/org/this-repo-name.git";
// const GIT_REPO_NAME = "this-repo-name"; // or an optional name
const cleantoken = new RegExp(`${GIT_TOKEN}`, 'g');
const clean = str => str.replace(cleantoken, '[private key]')
function runCommand(commandString) {
const options = {
stdio: 'pipe',
encoding: 'utf-8'
}
const [command, ...args] = commandString.match(/(".*?")|(\S+)/g);
const cmd = spawnSync(command, args, options);
console.log(
`$ ${clean(commandString)}
${clean(cmd.stdout)}`
)
// Error on status
if (cmd.status !== null && cmd.status !== 0){
console.log(`${clean(cmd.stderr)}`)
throw new Error(`Git backup failed`);
}
}
// clone the repository and set it as the cwd
// sadly, clone doesn't support --porcelain
const repoPath = path.join(process.cwd(), GIT_REPO_NAME)
if (fs.existsSync(repoPath)) {
// already checked out and cached into build
console.log(`Already exists: ${repoPath}`)
} else {
console.log(`Cloning into ${GIT_REPO_NAME}`)
runCommand(`git clone --quiet https://${GIT_USERNAME}:${GIT_TOKEN}@${GIT_REPO_URL} ${GIT_REPO_NAME}`)
}
process.chdir(repoPath)
runCommand(`ls -l`)
// setup the origin for the repository with personal access token
runCommand(`git remote rm origin`)
runCommand(`git remote add origin https://${GIT_USERNAME}:${GIT_TOKEN}@${GIT_REPO_URL}`)
runCommand(`git remote -v`)
// update local git config with email and username (required)
runCommand(`git config --local user.email ${GIT_EMAIL}`);
runCommand(`git config --local user.name ${GIT_USERNAME}`);
// setup the branch to HEAD
runCommand(`git checkout master`)
runCommand(`git pull origin master`)
// copy data command(s) here
runCommand(`cp -a ../site/data/. site/data/`)
// stage local files
runCommand(`git status --porcelain`)
runCommand(`git add .`);
// commit changes
const roboMessage = `commit by :robot: [skip ci]`
runCommand(`git commit -m "${roboMessage}"`);
// push to branch
runCommand(`git push --porcelain --set-upstream origin master`);
I’d be interested in Netlify’s take on what I’m doing here to see if this is recommended or there is an easy answer to your question. Sorry for all the edits. Been a long time since I worked on this solution.
-T