[Need Help] Drastic increase in build times for a gatsby site with gatsby-source-shopify Plugin

We are facing a critical issue with our Gatsby application using the gatsby-source-shopify plugin, particularly during the ‘source shopify variants’ stage of the build process, resulting in significantly prolonged build times. This problem is severely hampering our development and deployment efficiency.

Netlify Site Name: wizardly-brown-39c216.netlify.app

  • The ‘source shopify products’ stage completes in a reasonable timeframe (around 34-47 seconds for 253 nodes).
  • The ‘source shopify variants’ stage, however, is extremely slow, taking nearly 30 minutes (1777.314 seconds) for 645 nodes.
  • This issue started appearing recently and is consistent across deployments and local builds.

Attempts to Resolve:

  • Initial setup: Gatsby 4.25.7, gatsby-source-shopify 7.14.0, gatsby-source-graphql 4.9.0.
  • Updated to: Gatsby 5.13.1, gatsby-source-shopify 8.13.0, gatsby-source-graphql 5.13.0.
  • Despite updates, the problem persists.

Our configuration for the gatsby-source-shopify plugin in gatsby-config.js is as follows:

{
  resolve: "gatsby-source-shopify",
  options: {
    password: process.env.SHOPIFY_ADMIN_PASSWORD,
    storeUrl: process.env.GATSBY_SHOPIFY_STORE_URL,
    shopifyConnections: ["collections"],
    prioritize: true,
    downloadImages: false,
  },
}
OS: macOS 14.2.1
CPU: (8) arm64 Apple M2
Node: 20.11.0
Yarn: 1.22.19
npm: 10.2.4
Browsers: Chrome 120.0.6099.234, Safari 17.2.1

This problem was not observed in earlier versions or builds until recently.

Relevant Netlify Logs
11:17:05 PM: success source shopify products - 44.167s - 253 nodes
11:46:43 PM: success source shopify variants - 1777.314s - 645 nodes
11:46:48 PM: success source shopify collections - 5.471s - 112 nodes
11:46:50 PM: success Checking for changed pages - 0.000s
11:46:50 PM: success source and transform nodes - 1829.728s
11:46:51 PM: success building schema - 0.993s

Environment:

System:OS: macOS 14.2.1
CPU: (8) arm64 Apple M2
Shell: 5.9 - /bin/zsh
Binaries:Node: 20.11.0 - ~/.nvm/versions/node/v20.11.0/bin/node
Yarn: 1.22.19 - ~/.nvm/versions/node/v16.13.2/bin/yarn
npm: 10.2.4 - ~/.nvm/versions/node/v20.11.0/bin/npm
Languages:Python: 3.10.12 - /Users/zach/.pyenv/shims/python
Browsers:Chrome: 120.0.6099.234
Safari: 17.2.1
npmPackages^5.13.1 => 5.13.1
    gatsby-plugin-algolia: ^1.0.3 => 1.0.3
    gatsby-plugin-eslint: ^4.0.2 => 4.0.4
    gatsby-plugin-google-gtag: ^5.13.0 => 5.13.0
    gatsby-plugin-google-tagmanager: ^5.13.0 => 5.13.0
    gatsby-plugin-image: ^3.13.0 => 3.13.0
    gatsby-plugin-manifest: ^5.13.0 => 5.13.0
    gatsby-plugin-netlify: ^5.1.1 => 5.1.1
    gatsby-plugin-nullish-coalescing-operator: ^1.0.1 => 1.0.1
    gatsby-plugin-offline: ^6.13.0 => 6.13.0
    gatsby-plugin-postcss: ^6.13.0 => 6.13.0
    gatsby-plugin-react-helmet: ^6.13.0 => 6.13.0
    gatsby-plugin-react-svg: ^3.1.0 => 3.3.0
    gatsby-plugin-sass: ^6.13.0 => 6.13.0
    gatsby-plugin-sharp: ^5.13.0 => 5.13.0
    gatsby-plugin-sitemap: ^6.13.0 => 6.13.0
    gatsby-source-filesystem: ^5.13.0 => 5.13.0
    gatsby-source-graphql: ^5.13.0 => 5.13.0
    gatsby-source-shopify: ^8.13.0 => 8.13.0
    gatsby-source-storyblok: ^7.0.4 => 7.0.4
    gatsby-transformer-remark: ^6.13.0 => 6.13.0
    gatsby-transformer-sharp: ^5.13.0 => 5.13.0

Config Flags

baseRelDir: true

Additional Information:
Netlify Build Logs
Successful Build (Slow):

6:17:05 AM EST: success source shopify products - 44.167s - 253 nodes
6:46:43 AM EST: success source shopify variants - 1777.314s - 645 nodes
6:46:48 AM EST: success source shopify collections - 5.471s - 112 nodes
Failed Build:
6:34:16 AM EST: success source shopify products - 34.202s - 253 nodes
6:51:24 AM EST: Failed during stage "building site": Command did not finish within the time limit

Netlify Logs:

We have already created a github issue but we never got a reply. We suspect this might be an issue with the plugin itself or our configuration. Any insights or guidance you can provide would be greatly appreciated.

Just to confirm, have you ruled out the possibility of the Shopify API being slow to respond?

Yes, I have tried Shopify’s Bulk API explicitly and it was not slow at all!