Gatsby WordPress builds overnight now taking 10x longer and timing out

I have a WP/Gatsby site:

The builds were taking only 2/3 mins in July and without any changes, now in August, they are taking so long that they are timing out. Again, I changed nothing about the build and haven’t added any new content to WP back-end but the site will not build now. Please help!

Here is the last clean build that worked on the 27th of July:

Here is a build that was kicked off by something other than me (I pushed no changes and did not build the site manually) on the 31st of July:

Every build I’ve tried since this one has timed out. Notice how before this build every build ran in 2/3 mins. Now every build times out. I was to stress that I didn’t change anything between these two builds.

Hey @wanna! :wave:t2:
Welcome to The Community! :slight_smile:

Looking through your two logs there’s a couple of things that stand out. For starters, your latter log isn’t catching the Gatsby Cache so it’ll need to build from scratch, which undoubtedly takes more time

6:43:23 PM: ┌────────────────────────────────────────────────────────┐
6:43:23 PM: │ 1. onPreBuild command from netlify-plugin-gatsby-cache │
6:43:23 PM: └────────────────────────────────────────────────────────┘
6:43:23 PM: ​
6:43:24 PM: No Gatsby cache found. Building fresh.
6:43:24 PM: ​
6:43:24 PM: (netlify-plugin-gatsby-cache onPreBuild completed in 278ms)

Beyond that, your GraphQL queries in the earlier builds took 14 seconds

3:04:15 PM: success run queries - 14.822s - 4/4 0.27/s

but in the latter build took 1m 15s:

6:45:11 PM: success run queries - 85.683s - 56/56 0.65/s

So that’s a major issue, but could be due to just running more queries (since you had no cache) - could be due to network latencies (how’s your WP server doing? What’s its bandwidth capabilities?) - could be due to just having more data to churn through… did the site add a ton of content recently?

Beyond that, your earlier builds aren’t generating image thumbnails (or maybe they just didn’t need to? if the content was all cached and unchanged? Unclear without seeing more code and/or more builds over time?) but the latter build is trying to generate thumbnails… and that takes a long time

7:02:37 PM: success Generating image thumbnails - 1132.330s - 1626/1626 1.44/s

Over 18 minutes if my math’s correct… is this a new feature you added? That’s a lot of time.

What’s unfortunate is that your build basically finished but it was just after the timeout! Maybe a few more seconds and it should’ve deployed :thinking:

7:02:42 PM: ┌─────────────────────────────┐
7:02:42 PM: │   Netlify Build Complete    │
7:02:42 PM: └─────────────────────────────┘
7:02:42 PM: ​
7:02:42 PM: (Netlify Build completed in 19m 19.1s)
7:02:42 PM: Execution timed out after 15m0s
7:02:42 PM: Error running command: Command did not finish within the time limit
7:02:42 PM: Failing build: Failed to build site
7:02:42 PM: Failed during stage 'building site': Command did not finish within the time limit
7:02:42 PM: Finished processing build request in 21m3.315654519s

Each successive build has taken even longer. You are right that the queries took longer. Every previous build was generating thumbnails (unless cached) and the queries didn’t change. That’s what I am saying. I didn’t kick off this failing build AND nothing changed in the code between this build and previous builds that took under 5 mins. The WP server is on EC2 and has never sent an alert or been over 30% utilization in the last 6 weeks. What HAS changed is the number of images being pulled down.

Here is a build that went through without cache a couple weeks earlier. Queries were faster but there are less thumbnails.

I am trying 2 things: 1 - deleted every WP generated thumbnail size so that less media is pulled down. 2 - Upgrading to gastby-source-wordpress-expirimental which is supposed to be faster and have better caching on the server.

I cannot find any way to prevent Gatsby from fetching every image in the media folder… I know that is not a Netlify issue.

Hi, @wanna, it sounds like this is no longer a question about Netlify. As such, our support team likely won’t have much input on this.

However, if there are questions relating to Netlify, please let us know and we will do our best to answer.

If you do find the solution to get Gatsby to only fetch specific files and want to share that solution here, I’m sure other people searching this community site would find that very helpful.