Running a script that updates a file before build

I have a Jekyll site on Netlify and would like to be able to have the following workflow, but not sure how (or the simplest way) to accomplish:

  1. Update a CSV file in my repo (commit / push to master)
  2. Have Netlify run an NPM script I’ve created that creates some json files based on that updated csv and saves them to the _data dir (prior to building the site)
  3. Have Netlify build my Jekyll app including the newly created json files

I know how (and have successfully) ran the NPM script prior to Netlify build, but of course the changes to the json files are not committed/pushed to the repo, so the subsequent build does not have them…

So is it possible/recommended that I commit / push that change from within Netlify (there by triggering another build)? Or is there a better solution to this problem?

Shouldn’t be a problem. What have you tried? I’d do something like this, at a high level, as a build command:

  1. we clone your repo, and run bundle install automatically,
  2. then we run your build command:
    node ./make-my-json.js && jekyll build
  3. ???
  4. Profit :slight_smile:
1 Like

Thanks for the response. I’ve tried this:

In my package.json:

    "scripts": {
        "prebuild": "npm run gen-json-records",
        "build": "jekyll build --verbose --incremental",
        "gen-json-records": "node generateJsonRecords.js"

and in my netlify.toml file:

      command = "npm run build"

This seems to work (as in the script runs) but the build does not seem to include the updated files for some reason. I was thinking because the changes were not actually committed to the repo.

But perhaps my script is running before Netlify clones the repo so it’s being overwritten by the cloned repo? How can I run a script before jekylll build but after Netlify clones my repo?

Exactly the way I already showed you :slight_smile: Use a command like “node ./make-my-json.js && jekyll build”. That will work without npm scripts; I use it on tons of sites.

You can debug the build locally using the workflow here if you’d rather try it on your computer in our environment:

That will let you “see” what would be deployed to understand what files are where!

Thanks very much! I didn’t realize that what I had was actually working the whole time :man_facepalming:. My test of it working was just flawed.

Anyway, appreciate your help! :grin:

1 Like

So can you also run larger scripts such as:
cd dir && touch newfile.js && echo "module.exports = {some configurations here}" >> newfile.js && cd .. && npm run build ?

Also, how do you handle secret management in Netlify?

1 Like

Hey @sschipke, yep, something like that should work. To get really fancy with it, you may be interested in checking out our new (beta) Build Plugins feature:

As for secrets, I think this is probably a good place to start:

Want to check that out and let us know if you have any other questions?