Does Netlify store build file structure that's available on subsequent builds?

Support
, , ,
#1

Hey all I had sort of a basic question about the build environment/process and whether what I am doing locally would work on Netlify. I am looking to accomplish incremental builds by:

  1. During build save the a build timestamp in a local file
  2. Build outputs static html files in build directory generated from react scripts (similar to Gatsby)
  3. On subsequent build, rename build to cache, find all CMS data changed after the last build’s timestamp, generate new HTML only for those changed pages, copy unchanged HTML from cache to ‘build’, delete cache.

Hopefully that makes sense. It requires there to be an updated build directory and timestamp file present within the project structure before build. This works fine locally, but I’m not clear how these files would be stored on Netlify and/or how it would work with continuous deployment integration with git. Just wanted to get a more firm understanding here. Thanks.

#2

Hey @kmcaloon,

Please take what I say with a pinch of salt. I’m on mobile so I’m trying to cast my mind back!

I remember watching a podcast with @philhawksworth where they dubbed functionality like this as “endgame” — and I’d agree.

I believe that, in the meantime, this may be of use if you’re building with Gatsby though I can’t say that I’ve made use of it.

2 Likes
#3

@Pie’s advice matches mine, and I am somewhat of an expert :slight_smile: It won’t happen unless you make it happen with a plugin like that, or https://www.npmjs.com/package/cache-me-outside - and then you DO need to manage the cache restoring and comparing yourself. Also, make sure that your build can failover when there is no cache, since that will happen sometimes - we try super hard to keep the cache available, but there can be an occasional build without it, so if your build fails without it, you’ll get stuck in the fail state in that case if you REQUIRE it at build time.

2 Likes
#4

Thanks guys. @fool, that looks pretty interesting and looks somewhat similar to what I was looking for, however I am still a little unclear as far as what the current directory structure on Netlify would look like at the build/prebuild step.

For instance, say I have Netlify deployments set up via Github repo. That repo would just contain source code without any build or cache directories. Incremental builds would rely on both of these to run its logic. Now say a build were triggered via a commit to Github and Netlify begins the build process. While building on Netlify, will this caching plugin or custom node scripts have the prior build directory available to run the necessarily logic?

#5

The cache-me-outside package allows you to set what folders to include in the build cache and will be restored to the same place at the start of the build. You shouldn’t need to know where the folder/files get cache to, since they’ll be where they were on the previous build. Does that help?

#6

Thanks @Dennis I think that answers my question. My fear was after making a commit to the Github repo (which doesn’t have any build/cache files), Netlify would simply use the files from the repo to generate a new build without being able to leverage prior build/cache directories. So this definitely is not the case right?

#7

By default, that is exactly how our buildbot works. It does not cache build directories, only folders like node_modules. It would generate a new build based on your repository’s content without access to any previous build content.

My assertion is only true if you are using https://www.npmjs.com/package/cache-me-outside. This package is not used by default and you would have to configure it yourself. This package allows you to cache your build folder.

#8

Ahh got it that’s what I thought. Thanks for following up!