Incremental build deletes pages when it follows a branch build

Site name: eclectic-faun-bbae69

Hi there, I have an interesting issue with incremental building, coming from WordPress (using the GatsbyJS plugin, although another plugin had the same issue).

If I do a branch build via Git push, the next incremental build that happens from WordPress will delete all the existing pages and posts. It’s as if Netlify or Gatsby thought, since the data for those pages isn’t being sent, it doesn’t exist.

My build command is:
cross-env GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true gatsby build --verbose --log-pages

… and I do have gatsby-plugin-netlify installed.

One solution that would work for me is just to disable the incremental build when the webhook fires. It’s not a site with a huge amount of content.

Here’s the complete log below, you can see the “Deleted Pages” part. Thanks for any help or ideas!

8:56:15 AM: Build ready to start
8:56:19 AM: build-image version: 9289b698a503f132b265bd089ab40ce1d6f9e65e (focal)
8:56:19 AM: build-image tag: v4.12.0
8:56:19 AM: buildbot version: 8004afba0d79c2913deefb1ea0ee8f3ed710b828
8:56:19 AM: Fetching cached dependencies
8:56:19 AM: Starting to download cache of 246.2MB
8:56:20 AM: Finished downloading cache in 1.793974218s
8:56:20 AM: Starting to extract cache
8:56:23 AM: Finished extracting cache in 2.149811415s
8:56:23 AM: Finished fetching cache in 3.998508584s
8:56:23 AM: Starting to prepare the repo for build
8:56:23 AM: Preparing Git Reference refs/heads/main
8:56:23 AM: Parsing package.json dependencies
8:56:24 AM: Starting build script
8:56:24 AM: Installing dependencies
8:56:24 AM: Python version set to 2.7
8:56:24 AM: Started restoring cached node version
8:56:25 AM: Finished restoring cached node version
8:56:25 AM: v16.17.1 is already installed.
8:56:25 AM: Now using node v16.17.1 (npm v8.15.0)
8:56:25 AM: Started restoring cached build plugins
8:56:25 AM: Finished restoring cached build plugins
8:56:25 AM: Attempting ruby version 2.7.2, read from environment
8:56:26 AM: Using ruby version 2.7.2
8:56:26 AM: Using PHP version 8.0
8:56:27 AM: No npm workspaces detected
8:56:27 AM: Started restoring cached node modules
8:56:27 AM: Finished restoring cached node modules
8:56:27 AM: Started restoring cached go cache
8:56:27 AM: Finished restoring cached go cache
8:56:27 AM: Installing Go version 1.17 (requested 1.17)
8:56:32 AM: unset GOOS;
8:56:32 AM: unset GOARCH;
8:56:32 AM: export GOROOT='/opt/buildhome/.gimme/versions/go1.17.linux.amd64';
8:56:32 AM: export PATH="/opt/buildhome/.gimme/versions/go1.17.linux.amd64/bin:${PATH}";
8:56:32 AM: go version >&2;
8:56:32 AM: export GIMME_ENV="/opt/buildhome/.gimme/env/go1.17.linux.amd64.env"
8:56:32 AM: go version go1.17 linux/amd64
8:56:32 AM: Installing missing commands
8:56:32 AM: Verify run directory
8:56:34 AM: ​
8:56:34 AM: ────────────────────────────────────────────────────────────────
8:56:34 AM:   Netlify Build                                                 
8:56:34 AM: ────────────────────────────────────────────────────────────────
8:56:34 AM: ​
8:56:34 AM: ❯ Version
8:56:34 AM:   @netlify/build 27.20.1
8:56:34 AM: ​
8:56:34 AM: ❯ Flags
8:56:34 AM:   baseRelDir: true
8:56:34 AM:   buildId: 63404c1f26620811508d00a1
8:56:34 AM:   deployId: 63404c1f57ca9e000999721f
8:56:34 AM: ​
8:56:34 AM: ❯ Current directory
8:56:34 AM:   /opt/build/repo
8:56:34 AM: ​
8:56:34 AM: ❯ Config file
8:56:34 AM:   No config file was defined: using default values.
8:56:34 AM: ​
8:56:34 AM: ❯ Context
8:56:34 AM:   production
8:56:34 AM: ​
8:56:34 AM: ❯ Loading plugins
8:56:34 AM:    - @netlify/plugin-gatsby@3.4.7 from Netlify app
8:56:35 AM: ​
8:56:35 AM: ────────────────────────────────────────────────────────────────
8:56:35 AM:   1. @netlify/plugin-gatsby (onPreBuild event)                  
8:56:35 AM: ────────────────────────────────────────────────────────────────
8:56:35 AM: ​
8:56:35 AM: Found a Gatsby cache. We’re about to go FAST. ⚡️
8:56:35 AM: ​
8:56:35 AM: (@netlify/plugin-gatsby onPreBuild completed in 285ms)
8:56:35 AM: ​
8:56:35 AM: ────────────────────────────────────────────────────────────────
8:56:35 AM:   2. Build command from Netlify app                             
8:56:35 AM: ────────────────────────────────────────────────────────────────
8:56:35 AM: ​
8:56:35 AM: $ npm run build
8:56:35 AM: > wordpress-starter@0.1.0 build
8:56:35 AM: > cross-env GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true gatsby build --verbose --log-pages
8:56:35 AM: verbose 0.386057997 set gatsby_log_level: "verbose"
8:56:35 AM: verbose 0.38708108 set gatsby_executing_command: "build"
8:56:35 AM: verbose 0.387473701 loading local command from: /opt/build/repo/node_modules/gatsby/dist/commands/build.js
8:56:36 AM: verbose 1.421063582 running command: build
8:56:36 AM: verbose 1.421941188 Running build in "production" environment
8:56:37 AM: success compile gatsby files - 0.902s
8:56:37 AM: success load gatsby config - 0.017s
8:56:37 AM: warning Plugin gatsby-source-filesystem is not compatible with your gatsby version 4.24.1 - It requires gatsby@^2.2.0
8:56:37 AM: warning Plugin gatsby-transformer-sharp is not compatible with your gatsby version 4.24.1 - It requires gatsby@^2.12.1
8:56:38 AM: warning Plugin gatsby-plugin-manifest is not compatible with your gatsby version 4.24.1 - It requires gatsby@^2.25.0
8:56:38 AM: warning Plugin gatsby-source-filesystem is not compatible with your gatsby version 4.24.1 - It requires gatsby@^2.2.0
8:56:38 AM: warning Plugin gatsby-transformer-sharp is not compatible with your gatsby version 4.24.1 - It requires gatsby@^2.12.1
8:56:38 AM: warning Plugin gatsby-plugin-manifest is not compatible with your gatsby version 4.24.1 - It requires gatsby@^2.25.0
8:56:38 AM: success load plugins - 0.988s
8:56:38 AM: warning gatsby-plugin-react-helmet: Gatsby now has built-in support for modifying the document head. Learn more at https://gatsby.dev/gatsby-head
8:56:38 AM: success onPreInit - 0.006s
8:56:38 AM: success delete worker cache from previous builds - 0.002s
8:56:38 AM: success initialize cache - 0.026s
8:56:39 AM: success copy gatsby files - 0.089s
8:56:39 AM: success Compiling Gatsby Functions - 0.156s
8:56:39 AM: success onPreBootstrap - 0.171s
8:56:39 AM: verbose 3.643093736 Creating 15 worker
8:56:39 AM: success  gatsby-source-wordpress  ensuring plugin requirements are met - 0.074s
8:56:40 AM: success  gatsby-source-wordpress  diff schemas - 0.605s
8:56:40 AM: success  gatsby-source-wordpress  ingest WPGraphQL schema - 0.911s
8:56:40 AM: success createSchemaCustomization - 1.098s
8:56:41 AM: success  gatsby-source-wordpress  fetch root fields - 0.485s
8:56:44 AM: ⠀
8:56:44 AM: info  gatsby-source-wordpress  create kitten Clare, the “Soup Kitten” (#116)
8:56:44 AM: ⠀
8:56:44 AM: ⠀
8:56:44 AM: info  gatsby-source-wordpress  update user nabha (#1)
8:56:44 AM: ⠀
8:56:45 AM: ⠀
8:56:45 AM: info  gatsby-source-wordpress  create mediaItem DALL·E-2022-10-07-08.55.10-photo-of-a-kitten-covered-in-a-can-of-tomato-soup-covered-in-soup (#117)
8:56:45 AM: ⠀
8:56:45 AM: success  gatsby-source-wordpress  pull updates since last build - 3.773s
8:56:45 AM: success Downloading remote files - 2.265s - 1/1 0.44/s
8:56:45 AM: verbose 9.707837746 Checking for deleted pages
8:56:45 AM: verbose 9.708819942 Deleted 0 pages
8:56:45 AM: verbose 9.709127148 Found 0 changed pages
8:56:45 AM: success Checking for changed pages - 0.001s
8:56:45 AM: success source and transform nodes - 4.404s
8:56:46 AM: info Writing GraphQL type definitions to /opt/build/repo/.cache/schema.gql
8:56:48 AM: success building schema - 2.877s
8:56:48 AM: success createPages - 0.207s
8:56:48 AM: success createPagesStatefully - 0.302s
8:56:48 AM: info Total nodes: 99, SitePage nodes: 20 (use --verbose for breakdown)
8:56:48 AM: verbose 13.10202647 Number of node types: 11. Nodes per type: Directory: 1, File: 18, ImageSharp: 18, Site: 1, SiteBuildMetadata: 1, SitePage: 4, SitePlugin: 38, Wp: 1, WpKitten: 1, WpMediaItem: 15, WpUser: 1
8:56:48 AM: verbose 13.102122499 Checking for deleted pages
8:56:48 AM: verbose 13.103325554 Deleted 32 pages
8:56:48 AM: verbose 13.103631142 Found 1 changed page
8:56:48 AM: success Checking for changed pages - 0.000s
8:56:48 AM: success Cleaning up stale page-data - 0.085s
8:56:48 AM: success onPreExtractQueries - 0.001s
8:56:56 AM: success extract queries from components - 7.755s
8:56:56 AM: success  gatsby-source-wordpress  diff schemas - 0.000s
8:56:56 AM: success  gatsby-source-wordpress  ingest WPGraphQL schema - 0.000s
8:56:56 AM: success write out redirect data - 0.002s
8:56:56 AM: success Build manifest and related icons - 0.349s
8:56:56 AM: success onPostBootstrap - 0.351s
8:56:56 AM: info bootstrap finished - 21.302s
8:56:56 AM: success write out requires - 0.003s
8:56:58 AM: success Building production JavaScript and CSS bundles - 1.762s
8:56:58 AM: success  gatsby-source-wordpress  diff schemas - 2.032s
8:56:58 AM: success  gatsby-source-wordpress  ingest WPGraphQL schema - 2.081s
8:57:03 AM: success Building HTML renderer - 4.615s
8:57:03 AM: success Execute page configs - 0.031s
8:57:03 AM: success Caching Webpack compilations - 0.000s
8:57:03 AM: success run queries in workers - 0.025s - 3/3 119.72/s
8:57:03 AM: success Merge worker state - 0.001s
8:57:03 AM: success Rewriting compilation hashes - 0.000s
8:57:03 AM: success Writing page-data.json files to public directory - 0.001s - 1/1 860.88/s
8:57:11 AM: success Building static HTML for pages - 6.406s - 4/4 0.62/s
8:57:11 AM: success Delete previous page data - 0.004s
8:57:11 AM: info [gatsby-plugin-netlify] Creating SSR/DSG redirects...
8:57:11 AM: info [gatsby-plugin-netlify] Created 0 SSR/DSG redirects...
8:57:11 AM: success onPostBuild - 0.006s
8:57:12 AM: 
8:57:12 AM: Pages
8:57:12 AM: ┌ src/templates/kitten.js
8:57:12 AM: │ └   /kittens/clare-the-soup-kitten/
8:57:12 AM: ├ src/pages/404.js
8:57:12 AM: │ ├   /404/
8:57:12 AM: │ └   /404.html
8:57:12 AM: └ src/pages/index.js
8:57:12 AM:   └   /
8:57:12 AM:   ╭────────────────────────────────────────────────────────────────────╮
8:57:12 AM:   │                                                                    │
8:57:12 AM:   │     (SSG) Generated at build time                                  │
8:57:12 AM:   │   D (DSG) Deferred static generation - page generated at runtime   │
8:57:12 AM:   │   ∞ (SSR) Server-side renders at runtime (uses getServerData)      │
8:57:12 AM:   │   λ (Function) Gatsby function                                     │
8:57:12 AM:   │                                                                    │
8:57:12 AM:   ╰────────────────────────────────────────────────────────────────────╯
8:57:12 AM: info Done building in 36.630144835 sec
8:57:12 AM: info Built pages:
8:57:12 AM: Updated page: /404/
8:57:12 AM: Updated page: /404.html
8:57:12 AM: Updated page: /
8:57:12 AM: Updated page: /kittens/clare-the-soup-kitten/
8:57:12 AM: info Deleted pages:
8:57:12 AM: Deleted page: /kittens/penelope-the-architect/
8:57:12 AM: Deleted page: /kittens/mona-faces-her-fear/
8:57:12 AM: Deleted page: /kittens/joe-leaves-his-life-of-crime/
8:57:12 AM: Deleted page: /kittens/clarice-the-dancing-kitten/
8:57:12 AM: Deleted page: /kittens/the-famous-polly/
8:57:12 AM: Deleted page: /kittens/blues-cruise/
8:57:12 AM: Deleted page: /kittens/clarence-the-first-kitten-in-space/
8:57:12 AM: Deleted page: /kittens/horatios-best-friend/
8:57:12 AM: Deleted page: /kittens/grenadine-and-the-big-green-bug/
8:57:12 AM: Deleted page: /kittens/calico-and-the-climate/
8:57:12 AM: Deleted page: /kittens/aloicious-makes-a-difference/
8:57:12 AM: Deleted page: /kittens/the-adventure-of-zelda/
8:57:12 AM: Deleted page: /kittens/fernando-finds-a-spaceship/
8:57:12 AM: Deleted page: /contact/
8:57:12 AM: Deleted page: /about-the-kittens/
8:57:12 AM: Deleted page: /sample-page/
8:57:12 AM: ​
8:57:12 AM: (build.command completed in 36.9s)
8:57:12 AM: ​
8:57:12 AM: ────────────────────────────────────────────────────────────────
8:57:12 AM:   3. @netlify/plugin-gatsby (onBuild event)                     
8:57:12 AM: ────────────────────────────────────────────────────────────────
8:57:12 AM: ​
8:57:12 AM: Skipping Gatsby Functions and SSR/DSG support
8:57:12 AM: ​
8:57:12 AM: (@netlify/plugin-gatsby onBuild completed in 5ms)
8:57:12 AM: ​
8:57:12 AM: ────────────────────────────────────────────────────────────────
8:57:12 AM:   4. @netlify/plugin-gatsby (onPostBuild event)                 
8:57:12 AM: ────────────────────────────────────────────────────────────────
8:57:12 AM: ​
8:57:12 AM: Skipping Gatsby Functions and SSR/DSG support
8:57:12 AM: ​
8:57:12 AM: (@netlify/plugin-gatsby onPostBuild completed in 353ms)
8:57:12 AM: ​
8:57:12 AM: ────────────────────────────────────────────────────────────────
8:57:12 AM:   5. Deploy site                                                
8:57:12 AM: ────────────────────────────────────────────────────────────────
8:57:12 AM: ​
8:57:12 AM: Starting to deploy site from 'public'
8:57:12 AM: Creating deploy tree 
8:57:12 AM: Creating deploy upload records
8:57:12 AM: 11 new files to upload
8:57:12 AM: 0 new functions to upload
8:57:13 AM: Starting post processing
8:57:13 AM: Post processing - HTML
8:57:13 AM: Post processing - header rules
8:57:13 AM: Post processing - redirect rules
8:57:14 AM: Post processing done
8:57:17 AM: Site is live ✨
8:57:17 AM: Finished waiting for live deploy in 4.149092261s
8:57:17 AM: Site deploy was successfully initiated
8:57:17 AM: ​
8:57:17 AM: (Deploy site completed in 4.5s)
8:57:17 AM: ​
8:57:17 AM: ────────────────────────────────────────────────────────────────
8:57:17 AM:   6. @netlify/plugin-gatsby (onSuccess event)                   
8:57:17 AM: ────────────────────────────────────────────────────────────────
8:57:17 AM: ​
8:57:17 AM: ​
8:57:17 AM: (@netlify/plugin-gatsby onSuccess completed in 3ms)
8:57:17 AM: ​
8:57:17 AM: ────────────────────────────────────────────────────────────────
8:57:17 AM:   Netlify Build Complete                                        
8:57:17 AM: ────────────────────────────────────────────────────────────────
8:57:17 AM: ​
8:57:17 AM: (Netlify Build completed in 43.2s)
8:57:17 AM: Caching artifacts
8:57:17 AM: Started saving node modules
8:57:17 AM: Finished saving node modules
8:57:17 AM: Started saving build plugins
8:57:17 AM: Finished saving build plugins
8:57:17 AM: Started saving pip cache
8:57:17 AM: Finished saving pip cache
8:57:17 AM: Started saving emacs cask dependencies
8:57:17 AM: Finished saving emacs cask dependencies
8:57:17 AM: Started saving maven dependencies
8:57:17 AM: Finished saving maven dependencies
8:57:17 AM: Started saving boot dependencies
8:57:17 AM: Finished saving boot dependencies
8:57:17 AM: Started saving rust rustup cache
8:57:17 AM: Finished saving rust rustup cache
8:57:17 AM: Started saving go dependencies
8:57:17 AM: Finished saving go dependencies
8:57:17 AM: Build script success
8:57:18 AM: Uploading Cache of size 246.2MB
8:57:20 AM: Finished processing build request in 1m1.594339665s

Hi @nabha,

For Gatsby incremental builds to work, the pages should exist in Gatsby’s cache. Could you try to change your build command to something like ls $NETLIFY_CACHE_DIR which should list the contents of the cache folder. You might have to try multiple times till you get to the Gatsby’s cache folder to verify the contents there.