Gatsby 5.13.7 timeouts on Netlify, not local, only since a while!

Since a while we get massive timeouts with our Gatsby Wordpress GraphQL Website.
It was not always an issue!

The error is a timeout. (full build log at the end)

9:45:12 AM: error  gatsby-source-wordpress  Error: connect ETIMEDOUT xxx.xxx.xxx.xxx:443
9:45:12 AM:     at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1595:16)
9:45:12 AM:     at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17)

We updated all packages to the latest release, but we still get the error

"gatsby": "^5.13.7",
"gatsby-plugin-feed": "^5.13.1",
"gatsby-plugin-gdpr-cookies": "^2.0.9",
"gatsby-plugin-google-analytics": "^5.13.1",
"gatsby-plugin-image": "^3.13.1",
"gatsby-plugin-layout": "^4.13.1",
"gatsby-plugin-mailchimp": "^5.2.2",
"gatsby-plugin-manifest": "^5.13.1",
"gatsby-plugin-offline": "^6.13.2",
"gatsby-plugin-react-helmet": "^6.13.1",
"gatsby-plugin-remove-serviceworker": "^1.0.0",
"gatsby-plugin-sharp": "^5.13.1",
"gatsby-plugin-styled-components": "^6.13.1",
"gatsby-plugin-typography": "^5.13.1",
"gatsby-remark-copy-linked-files": "^6.13.1",
"gatsby-remark-prismjs": "^7.13.1",
"gatsby-remark-responsive-iframe": "^6.13.1",
"gatsby-remark-smartypants": "^6.13.1",
"gatsby-source-wordpress": "^7.13.4",
"gatsby-transformer-remark": "^6.13.1",
"gatsby-transformer-sharp": "^5.13.1",

whereby we also tried out different configurations for gatsby-source-wordpress,
which of none seem to have an impact on the build failing or not.

schema: {
  timeout: 720000,
  perPage: 10,
  requestConcurrency: 10,
  previewRequestConcurrency: 10
}

Wordpress runs on PHP 8.3.7 (Most current available, high max_execution_time of 720) with

WP Gatsby 2.3.3 (current)
WPGraphQL 1.28.0 (current)

We do not see any errors on localhost, not in dev nor in build.

We understand that the error seems to be server-side, but we do not fully understand this, as the server is running smooth in different matters.

Any help appreciated!!

Full build log

9:52:35 AM: build-image version: ecdc8b770f4a0193fd3f258c1bc6029e681813a4 (focal)
9:52:35 AM: buildbot version: 778f625377bd173cd306eb0b71ebeb25fdfff620
9:52:35 AM: Building without cache
9:52:35 AM: Starting to prepare the repo for build
9:52:36 AM: No cached dependencies found. Cloning fresh repo
9:52:36 AM: git clone --filter=blob:none git@bitbucket.org:fabian_w/website__von-bartha
9:52:36 AM: Preparing Git Reference refs/heads/master
9:52:38 AM: Custom build command detected. Proceeding with the specified command: 'npm run build'
9:52:39 AM: Starting to install dependencies
9:52:39 AM: Python version set to 3.8
9:52:39 AM: Attempting Ruby version 2.7.2, read from environment
9:52:39 AM: Using Ruby version 2.7.2
9:52:40 AM: Started restoring cached go cache
9:52:40 AM: Finished restoring cached go cache
9:52:41 AM: go version go1.19.13 linux/amd64
9:52:42 AM: Using PHP version 8.0
9:52:43 AM: Attempting Node.js version 'v20.11.0' from .nvmrc
9:52:43 AM: Downloading and installing node v20.11.0...
9:52:44 AM: Downloading https://nodejs.org/dist/v20.11.0/node-v20.11.0-linux-x64.tar.xz...
9:52:44 AM: Computing checksum with sha256sum
9:52:44 AM: Checksums matched!
9:52:46 AM: Now using node v20.11.0 (npm v10.2.4)
9:52:46 AM: Enabling Node.js Corepack
9:52:46 AM: Started restoring cached build plugins
9:52:46 AM: Finished restoring cached build plugins
9:52:46 AM: Started restoring cached corepack dependencies
9:52:46 AM: Finished restoring cached corepack dependencies
9:52:46 AM: No npm workspaces detected
9:52:46 AM: Started restoring cached node modules
9:52:46 AM: Finished restoring cached node modules
9:52:46 AM: Installing npm packages using npm version 10.2.4
9:53:25 AM: npm WARN deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
9:53:26 AM: npm WARN deprecated rimraf@3.0.2: Rimraf versions prior to v4 are no longer supported
9:53:26 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:26 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:26 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:26 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:26 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:26 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:26 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:27 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:27 AM: npm WARN deprecated @babel/plugin-proposal-nullish-coalescing-operator@7.18.6: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-nullish-coalescing-operator instead.
9:53:27 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:27 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:27 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:27 AM: npm WARN deprecated @babel/plugin-proposal-optional-chaining@7.21.0: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-optional-chaining instead.
9:53:27 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:27 AM: npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
9:53:28 AM: npm WARN deprecated rimraf@2.6.3: Rimraf versions prior to v4 are no longer supported
9:53:28 AM: npm WARN deprecated browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
9:53:28 AM: npm WARN deprecated browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
9:53:28 AM: npm WARN deprecated flatten@1.0.3: flatten is deprecated in favor of utility frameworks such as lodash.
9:53:29 AM: npm WARN deprecated circular-json@0.3.3: CircularJSON is in maintenance only, flatted is its successor.
9:53:30 AM: npm WARN deprecated stable@0.1.8: Modern JS already guarantees Array#sort() is a stable sort, so this library is deprecated. See the compatibility table on MDN: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#browser_compatibility
9:53:31 AM: npm WARN deprecated @babel/plugin-proposal-numeric-separator@7.18.6: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-numeric-separator instead.
9:53:31 AM: npm WARN deprecated workbox-google-analytics@4.3.1: It is not compatible with newer versions of GA starting with v4, as long as you are using GAv3 it should be ok, but the package is not longer being maintained
9:53:32 AM: npm WARN deprecated rimraf@2.7.1: Rimraf versions prior to v4 are no longer supported
9:53:32 AM: npm WARN deprecated @hapi/address@2.1.4: Moved to 'npm install @sideway/address'
9:53:32 AM: npm WARN deprecated @humanwhocodes/config-array@0.5.0: Use @eslint/config-array instead
9:53:32 AM: npm WARN deprecated @babel/plugin-proposal-class-properties@7.18.6: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-class-properties instead.
9:53:32 AM: npm WARN deprecated string-similarity@1.2.2: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.
9:53:32 AM: npm WARN deprecated caller-id@0.1.0: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.
9:53:32 AM: npm WARN deprecated @hapi/topo@3.1.6: This version has been deprecated and is no longer supported or maintained
9:53:32 AM: npm WARN deprecated @hapi/bourne@1.3.2: This version has been deprecated and is no longer supported or maintained
9:53:32 AM: npm WARN deprecated @babel/plugin-proposal-object-rest-spread@7.20.7: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-object-rest-spread instead.
9:53:33 AM: npm WARN deprecated glob@6.0.4: Glob versions prior to v9 are no longer supported
9:53:33 AM: npm WARN deprecated rimraf@2.6.3: Rimraf versions prior to v4 are no longer supported
9:53:33 AM: npm WARN deprecated @humanwhocodes/object-schema@1.2.1: Use @eslint/object-schema instead
9:53:33 AM: npm WARN deprecated typescript-styled-plugin@0.18.3: Deprecated in favor of https://github.com/styled-components/typescript-styled-plugin
9:53:33 AM: npm WARN deprecated browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
9:53:33 AM: npm WARN deprecated @hapi/hoek@8.5.1: This version has been deprecated and is no longer supported or maintained
9:53:34 AM: npm WARN deprecated @hapi/joi@15.1.1: Switch to 'npm install joi'
9:53:35 AM: npm WARN deprecated vscode-emmet-helper@2.6.4: This package has been renamed to @vscode/emmet-helper, please update to the new name
9:53:38 AM: npm WARN deprecated core-js@2.6.12: core-js@<3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js.
9:53:48 AM: added 2587 packages, and audited 2589 packages in 1m
9:53:48 AM: 497 packages are looking for funding
9:53:48 AM:   run `npm fund` for details
9:53:48 AM: 24 vulnerabilities (1 low, 15 moderate, 8 high)
9:53:48 AM: To address issues that do not require attention, run:
9:53:48 AM:   npm audit fix
9:53:48 AM: To address all issues possible (including breaking changes), run:
9:53:48 AM:   npm audit fix --force
9:53:48 AM: Some issues need review, and may require choosing
9:53:48 AM: a different dependency.
9:53:48 AM: Run `npm audit` for details.
9:53:48 AM: npm packages installed
9:53:49 AM: Successfully installed dependencies
9:53:49 AM: Starting build script
9:53:50 AM: Detected 1 framework(s)
9:53:50 AM: "gatsby" at version "5.13.7"
9:53:50 AM: Section completed: initializing
9:53:51 AM: ​
9:53:51 AM: Netlify Build                                                 
9:53:51 AM: ────────────────────────────────────────────────────────────────
9:53:51 AM: ​
9:53:51 AM: ❯ Version
9:53:51 AM:   @netlify/build 29.53.0
9:53:51 AM: ​
9:53:51 AM: ❯ Flags
9:53:51 AM:   baseRelDir: true
9:53:51 AM:   buildId: 66bdb3b93f6fbf81b7c84823
9:53:51 AM:   deployId: 66bdb3b93f6fbf81b7c84825
9:53:51 AM: ​
9:53:51 AM: ❯ Current directory
9:53:51 AM:   /opt/build/repo
9:53:51 AM: ​
9:53:51 AM: ❯ Config file
9:53:51 AM:   /opt/build/repo/netlify.toml
9:53:51 AM: ​
9:53:51 AM: ❯ Context
9:53:51 AM:   production
9:53:51 AM: ​
9:53:51 AM: ❯ Loading plugins
9:53:51 AM:    - @netlify/plugin-gatsby@3.8.1 from netlify.toml and package.json
9:53:53 AM: Skipping @netlify/plugin-gatsby work, because used Gatsby version supports adapters.
9:53:53 AM: ​
9:53:53 AM: build.command from netlify.toml                               
9:53:53 AM: ────────────────────────────────────────────────────────────────
9:53:53 AM: ​
9:53:53 AM: $ npm run build
9:53:53 AM: > von-bartha@2.0.0 build
9:53:53 AM: > gatsby build
9:53:55 AM: success compile gatsby files - 1.398s
9:53:55 AM: success load gatsby config - 0.026s
9:53:58 AM: success Installing Netlify adapter (gatsby-adapter-netlify@^1.0.4) - 2.342s
9:53:58 AM: info If you plan on staying on this deployment platform, consider installing "gatsby-adapter-netlify@^1.0.4" as a dependency in your project. This will give you faster and more robust installs.
9:53:58 AM: info Using gatsby-adapter-netlify adapter
9:53:58 AM: warning Plugin gatsby-plugin-gdpr-cookies is not compatible with your gatsby version 5.13.7 - It requires gatsby@^2.24.77 || ^3.0.0 || ^4.0.0
9:53:58 AM: warning Plugin gatsby-plugin-gdpr-cookies is not compatible with your gatsby version 5.13.7 - It requires gatsby@^2.24.77 || ^3.0.0 || ^4.0.0
9:53:59 AM: success load plugins - 0.665s
9:53:59 AM: warning gatsby-plugin-react-helmet: Gatsby now has built-in support for modifying the document head. Learn more at https://gatsby.dev/gatsby-head
9:53:59 AM: success onPreInit - 0.003s
9:53:59 AM: success initialize cache - 0.042s
9:53:59 AM: success copy gatsby files - 0.054s
9:53:59 AM: success Compiling Gatsby Functions - 0.167s
9:53:59 AM: success onPreBootstrap - 0.176s
9:58:20 AM: error  gatsby-source-wordpress  Error: connect ETIMEDOUT xxx.xxx.xxx.xxx:443
9:58:20 AM:     at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1595:16)
9:58:20 AM:     at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17)
9:58:20 AM: GraphQL request to https://backend.vonbartha.com/graphql failed.
9:58:20 AM: Please ensure the following statements are true
9:58:20 AM:   - your WordPress URL is correct in gatsby-config.js
9:58:20 AM:   - your server is responding to requests
9:58:20 AM:   - WPGraphQL and WPGatsby are installed and active in your WordPress backend
9:58:20 AM:   - Your WordPress debug.log does not contain critical errors
9:58:20 AM: not finished createSchemaCustomization - 261.089s
9:58:20 AM: not finished  gatsby-source-wordpress  ensuring plugin requirements are met - 261.076s
9:58:21 AM: ​
9:58:21 AM: "build.command" failed                                        
9:58:21 AM: ────────────────────────────────────────────────────────────────
9:58:21 AM: ​
9:58:21 AM:   Error message
9:58:21 AM:   Command failed with exit code 1: npm run build (https://ntl.fyi/exit-code-1)
9:58:21 AM: ​
9:58:21 AM:   Error location
9:58:21 AM:   In build.command from netlify.toml:
9:58:21 AM:   npm run build
9:58:21 AM: ​
9:58:21 AM:   Resolved config
9:58:21 AM:   build:
9:58:21 AM:     command: npm run build
9:58:21 AM:     commandOrigin: config
9:58:21 AM:     environment:
9:58:21 AM:       - API_URL
9:58:21 AM:       - DEPLOY_ENV
9:58:21 AM:       - GATSBY_FRONTEND_URL
9:58:21 AM:       - GATSBY_GIT_BRANCH
9:58:21 AM:       - NPM_FLAGS
9:58:21 AM:     publish: /opt/build/repo/public
9:58:21 AM:     publishOrigin: config
9:58:21 AM:   plugins:
9:58:21 AM:     - inputs: {}
9:58:21 AM:       origin: config
9:58:21 AM:       package: '@netlify/plugin-gatsby'
9:58:22 AM: Failed during stage 'building site': Build script returned non-zero exit code: 2 (https://ntl.fyi/exit-code-2)
9:58:22 AM: Build failed due to a user error: Build script returned non-zero exit code: 2
9:58:22 AM: Failing build: Failed to build site
9:58:22 AM: Finished processing build request in 5m46.805s

We found that Netlify is using different IPs to connect to our server.
How high are the chances that some of them are blocked?
Because when a timeout happens, we even cannot see them in our logs on the shared env…

Did you try contacting your GraphQL server? Connection timedout is an issue you need to sort with your API server, not Netlify.

Hi,

thanks for the reply.
Yes we did contact the server, everything works fine from our machines.

Most likely our host has a blacklist for a range of IP addresses that are blocked and this firewall causes the timeout. If we assume correctly Netlify is using different servers / IP addresses per build, the only way to bypass this is to use the private-connectivity feature from the enterprice plan. Is this assumption correct?
On the other side: Does Netlify provide a list of ranges for IP Adresses that we can use to whitelist?

That is correct.

That is exactly what the Private Connectivity feature is.

So without an enterprise account with private connectivity feature we are not able to get a list of IPs?
Or is it possible to stay in our current plan and just add this feature?

No, it’s limited to Enterprise plan at the moment.

Thanks for the help.
Unfortunately that’s too expensive for us.

We contacted our hosting provider and they asked for a List of IP Adresses Netlify uses.
Is it possible to get our hands on such a list @hrishikesh?
Further, is it possible to see which IP was used for a build so we can test whitelisting?

We urgently need the IP Adress of each build to help our backend host debug the problem.
How can we get it?

Answered here: Obtain the IP Address of the server who runs the build command