Gatsby v4 works locally, but timed out on Netlify

ajmalafif · October 27, 2021, 2:58pm

hey team,

I finally managed to migrate my current repo to Gatsby v4 and it compiles locally, but times out on Netlify. I‘ve tried a couple of suggestions I‘ve found on Google:

Installing netlify-plugin-gatsby
set AWS_LAMBDA_JS_RUNTIME to nodejs14.x in Netlify UI

So far nothing works. Here‘s the Netlify deploy build log.

Is it because gastby v4 is not officially supported yet or I am missing something important? Any pointer or suggestion is greatly appreciated, thank you!

stevepepple · October 28, 2021, 3:09pm

I’m also seeing this same issue. The build times out no the Building production JavaScript and CSS bundles step with this message:

success run queries in workers - 438.798s - 1795/1795 4.09/s
8:00:06 AM: error UNHANDLED EXCEPTION write EPIPE
8:00:06 AM: 
8:00:06 AM: 
8:00:06 AM:   Error: write EPIPE
8:00:06 AM:   
8:00:06 AM:   - child_process.js:841 ChildProcess.target._send
8:00:06 AM:     internal/child_process.js:841:20
8:00:06 AM:   
8:00:06 AM:   - child_process.js:712 ChildProcess.target.send
8:00:06 AM:     internal/child_process.js:712:19
8:00:06 AM:   
8:00:06 AM:   - index.js:290 WorkerPool.sendMessage
8:00:06 AM:     [site]/[gatsby-worker]/dist/index.js:290:19
8:00:06 AM:   
8:00:06 AM:   - worker-messaging.ts:22 
8:00:06 AM:     [site]/[gatsby]/src/utils/jobs/worker-messaging.ts:22:22
8:00:06 AM:   
8:00:06 AM: 
8:00:06 AM: not finished Merge worker state - 0.260s
8:00:06 AM:

The Gatsby team fixed the issue in build #4.0.1. That version works for me locally but times out on Netlify:

github.com/gatsbyjs/gatsby

Gatsby build occasionally time out after "Caching JavaScript and CSS webpack compilation"

opened 08:10AM - 17 Oct 21 UTC

closed 06:06AM - 27 Oct 21 UTC

robinmetral

type: bug status: confirmed

### Preliminary Checks - [X] This issue is not a duplicate. Before opening a …new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues - [X] This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions ### Description About a month ago, [builds of my Gatsby site](https://github.com/robinmetral/gatsby-source-s3/actions/workflows/test.yml) started failing in GitHub Actions. _For context, this Gatsby site is a minimal example using my [`gatsby-source-s3`](https://github.com/robinmetral/gatsby-source-s3) plugin, that I use for testing the plugin end-to-end._ Here's from an [example of a failed build](https://github.com/robinmetral/gatsby-source-s3/runs/3912139392?check_suite_focus=true): ```sh # ℹ️ omitted earlier logs, check out the link above to view the entire output success run page queries - 13.660s - 1/1 0.07/s # 👈 this is after all nodes were sourced from S3, so I don't think the problem is with my plugin success write out requires - 0.011s success Building production JavaScript and CSS bundles - 9.577s success Rewriting compilation hashes - 0.005s success Writing page-data.json files to public directory - 0.017s - 1/1 57.50/s success Caching JavaScript and CSS webpack compilation - 13.059s # 👈 this is where is hangs Error: Timed out waiting for: http://localhost:9000 # 👈 this is the timeout from cypress. I've also increased it to see if it just took too long, but that was not the issue ``` At first, all builds were failing, and in the last week it seems that some of them are passing again while others are still failing. I'm not sure what's happening here, as there are no helpful logs in the build output. A local `gatsby develop` works completely fine. The occasional passing build seem to rule out issues in my project or in other dependencies. I also [asked on discord](https://discord.com/channels/484383807575687178/537691356487876624/898110109966811156) but it didn't seem like anyone was having the same issue. Any thoughts on: - what the issue is here - how I could debug this—I haven't found anything on [the docs](https://www.gatsbyjs.com/docs/reference/gatsby-cli/#build) about this Thank you! _A note about the reproduction link 👇 I am aware that the issue template says "Do not link to your actual project", but in my case this example site is exactly a minimal example, since the only thing is does is pulling images from S3. I added the GH Action output there, [here's the link to the actual source](https://github.com/robinmetral/gatsby-source-s3/tree/main/examples/gatsby-starter-source-s3) although I doubt that it will be helpful._ ### Reproduction Link https://github.com/robinmetral/gatsby-source-s3/runs/3912139392?check_suite_focus=true ### Steps to Reproduce 1. Visit the link above to see the failing build output. I cannot reliably reproduce this locally, as mentioned in the summary. ### Expected Result `gatsby build` should not time out ### Actual Result `gatsby build` is flaky and unreliable, and regularly hangs in CI envs ### Environment My local environment is not the issue here, this is failing on a clean GH Actions machine with: - Ubuntu 20.04 LTS - Latest Node 14 - Gatsby 3 ### Config Flags _No response_

ajmalafif · October 29, 2021, 5:15pm

seems like so many things are still not working with v4 especially the ones with community plugins

perry · November 1, 2021, 5:18pm

hi there @ajmalafif - sorry to hear that there are still some rough edges. Please do keep continuing to file issues - that is the best way for the team that works on these specific things to learn about bugs you are experiencing so they can work on them efficiently! thank you.

stevepepple · November 2, 2021, 5:12pm

I’m happy to file an issue and logs. Where do you want it to be reported?

perry · November 2, 2021, 9:28pm

here please! if it seems like its an issue with our plugin:

djben · November 10, 2021, 4:01pm

I’m seeing this same ‘write EPIPE’ error, referenced here with Gatsby (they should probably close this issue, I only responded there as it was the first place I saw it):

github.com/gatsbyjs/gatsby

Gatsby 4 - EPIPE error on Netlify

opened 06:10PM - 29 Oct 21 UTC

closed 08:54AM - 17 Nov 21 UTC

SamHoltby

type: bug status: needs reproduction

### Preliminary Checks - [X] This issue is not a duplicate. Before opening a …new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues - [X] This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions ### Description Just upgraded to Gatsby 4 and everything is working fine locally. I can npm develop and build without issues. However, as soon as I push to Netlify the below error is returned near the end of building. ``` 4:07:59 PM: success run queries in workers - 158.327s - 621/621 3.92/s 4:16:57 PM: error UNHANDLED EXCEPTION write EPIPE 4:16:57 PM: 4:16:57 PM: 4:16:57 PM: Error: write EPIPE 4:16:57 PM: 4:16:57 PM: - child_process:846 ChildProcess.target._send 4:16:57 PM: node:internal/child_process:846:20 4:16:57 PM: 4:16:57 PM: - child_process:719 ChildProcess.target.send 4:16:57 PM: node:internal/child_process:719:19 4:16:57 PM: 4:16:57 PM: - index.js:290 WorkerPool.sendMessage 4:16:57 PM: [repo]/[gatsby-worker]/dist/index.js:290:19 4:16:57 PM: 4:16:57 PM: - worker-messaging.ts:22 4:16:57 PM: [repo]/[gatsby]/src/utils/jobs/worker-messaging.ts:22:22 4:16:57 PM: 4:16:57 PM: 4:16:57 PM: not finished Merge worker state - 0.168s ``` The Netlify site is deploying on node v16.13.0 (npm v8.1.0) - which is the same as my local environment(Windows 10). Netlify support believe it's an issue with Gatsby, which surely can't be the case if it's working locally? ### Reproduction Link Unable to provide ### Steps to Reproduce 1. Install the following packages after cloning https://github.com/gatsbyjs/gatsby-starter-wordpress-blog : gatsby: ^4.0.2 => 4.0.2 gatsby-background-image: ^1.5.3 => 1.5.3 gatsby-plugin-feed: ^4.0.0 => 4.0.0 gatsby-plugin-google-tagmanager: ^4.0.0 => 4.0.0 gatsby-plugin-image: ^2.0.0 => 2.0.0 gatsby-plugin-manifest: ^4.0.0 => 4.0.0 gatsby-plugin-nprogress: ^4.0.0 => 4.0.0 gatsby-plugin-offline: ^5.0.0 => 5.0.0 gatsby-plugin-react-helmet: ^5.0.0 => 5.0.0 gatsby-plugin-react-svg: ^3.0.1 => 3.1.0 gatsby-plugin-remove-fingerprints: ^0.0.2 => 0.0.2 gatsby-plugin-sass: ^5.0.0 => 5.0.0 gatsby-plugin-sharp: ^4.0.1 => 4.0.1 gatsby-plugin-sitemap: ^5.0.0 => 5.0.0 gatsby-plugin-styled-components: ^5.0.0 => 5.0.0 gatsby-plugin-svgr-loader: ^0.1.0 => 0.1.0 gatsby-plugin-web-font-loader: ^1.0.4 => 1.0.4 gatsby-source-filesystem: ^4.0.0 => 4.0.0 gatsby-source-wordpress: ^6.0.0 => 6.0.0 gatsby-transformer-sharp: ^4.0.0 => 4.0.0 2. Setup WordPress installation and configure with the following gatsby-config: ``` { resolve: `gatsby-source-wordpress`, options: { url: process.env.GRAPHQL_API, debug: { graphql: { writeQueriesToDisk: true, }, }, production: { allow404Images: true, }, auth: process.env.NODE_ENV === "production" ? { htaccess: { username: process.env.HTACCESS_USERNAME, password: process.env.HTACCESS_PASSWORD, }, } : {}, type: { Post: { limit: process.env.NODE_ENV === `development` ? 30 : process.env.POST_LIMIT || 999999, }, Comment: { exclude: true, }, PostFormat: { exclude: true, }, }, }, }, ``` 3. Deploy to Netlify ### Expected Result Site builds successfully ### Actual Result Site build fails each time with `UNHANDLED EXCEPTION write EPIPE` error. ### Environment ```shell My local environment: System: OS: Windows 10 10.0.19042 CPU: (16) x64 Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz Binaries: Node: 16.13.0 - C:\Program Files\nodejs\node.EXE npm: 8.1.0 - C:\Program Files\nodejs\npm.CMD Languages: Python: 3.10.0 Browsers: Edge: Spartan (44.19041.1266.0), Chromium (95.0.1020.30) npmPackages: gatsby: ^4.0.2 => 4.0.2 gatsby-background-image: ^1.5.3 => 1.5.3 gatsby-plugin-feed: ^4.0.0 => 4.0.0 gatsby-plugin-google-tagmanager: ^4.0.0 => 4.0.0 gatsby-plugin-image: ^2.0.0 => 2.0.0 gatsby-plugin-manifest: ^4.0.0 => 4.0.0 gatsby-plugin-nprogress: ^4.0.0 => 4.0.0 gatsby-plugin-offline: ^5.0.0 => 5.0.0 gatsby-plugin-react-helmet: ^5.0.0 => 5.0.0 gatsby-plugin-react-svg: ^3.0.1 => 3.1.0 gatsby-plugin-remove-fingerprints: ^0.0.2 => 0.0.2 gatsby-plugin-sass: ^5.0.0 => 5.0.0 gatsby-plugin-sharp: ^4.0.1 => 4.0.1 gatsby-plugin-sitemap: ^5.0.0 => 5.0.0 gatsby-plugin-styled-components: ^5.0.0 => 5.0.0 gatsby-plugin-svgr-loader: ^0.1.0 => 0.1.0 gatsby-plugin-web-font-loader: ^1.0.4 => 1.0.4 gatsby-source-filesystem: ^4.0.0 => 4.0.0 gatsby-source-wordpress: ^6.0.0 => 6.0.0 gatsby-transformer-sharp: ^4.0.0 => 4.0.0 npmGlobalPackages: gatsby-cli: 4.0.0 Netlify site: node v16.13.0 (npm v8.1.0) Using ruby version 2.7.1 Using PHP version 8.0 ``` ### Config Flags _No response_

You’ll see in my comment there, I validated a theory posted that this is a memory or timeout issue.

If I drastically reduce the static content of my site, Netlify deploys it just fine.

Can someone at Netlify try bumping up my build timeout and/or memory limit?

My Netlify instance title is amazing-wiles-1918a6

hrishikesh · November 10, 2021, 4:46pm

Unfortunately, memory limit cannot be changed. We can bump the build time, but since it’s not a build time issue, I don’t see the benefit in doing this.

This might shed more info:

djben · November 10, 2021, 5:05pm

Maybe you can just bump my build time for now? You can see in my deploy history I have successful vs. unsuccessful deploys with V4 – maybe there is a log to check for memory usage differences.

I’ll also work on trying to pull down the Netlify docker image and checking memory usage myself.

hrishikesh · November 10, 2021, 6:48pm

Sure, the time limit has been bumped to 30 mins for now.

Unfortunately, we don’t log the build memory usage yet, so we don’t have that data.

djben · November 11, 2021, 3:10am

So I went through the process for the first time of using Netlify’s build-image as @hrishikesh suggested.

It took a little bit of work and I ran into a bug where the submodule for all of my static assets wasn’t pulling down properly (Submodules do not get copied on cloning to container · Issue #444 · netlify/build-image · GitHub), but upon manually copying into assets to the build image volume in the right folder, I was able to start building my app.

With the default memory limit of 3GB, I seem to get the same write EPIPE error and my build fails (good, I reproduced it!).

I then bumped by Docker memory limit to 4GB and… the build succeeds.

I guess the question now is deciding how I move forward if the 3GB limit for Netlify builds cannot be changed

djben · February 27, 2022, 2:56pm

Did the memory limit get reduced below 3GB recently, or were there any other changes to the build environments recently?

I have not been able to run a deploy of my app since February 14th, despite reducing it’s static content to the bare minimum

homearanya · March 1, 2022, 10:53am

Definitely something has changed on Netlify since Feb 14th. I had to downgrade to Gatsby v3. Same old issue on Gatsby v4

8:20:45 PM: success run queries in workers - 42.674s - 478/478 11.20/s
8:21:55 PM: success Running gatsby-plugin-sharp.IMAGE_PROCESSING jobs - 111.872s - 270/270 2.41/s
8:21:55 PM: error UNHANDLED EXCEPTION write EPIPE
8:21:55 PM: 
8:21:55 PM: 
8:21:55 PM:   Error: write EPIPE
8:21:55 PM:   
8:21:55 PM:   - child_process.js:841 ChildProcess.target._send
8:21:55 PM:     internal/child_process.js:841:20
8:21:55 PM:   
8:21:55 PM:   - child_process.js:712 ChildProcess.target.send
8:21:55 PM:     internal/child_process.js:712:19
8:21:55 PM:   
8:21:55 PM:   - index.js:291 WorkerPool.sendMessage
8:21:55 PM:     [repo]/[gatsby-worker]/dist/index.js:291:19
8:21:55 PM:   
8:21:55 PM:   - worker-messaging.ts:22 
8:21:55 PM:     [repo]/[gatsby]/src/utils/jobs/worker-messaging.ts:22:22
8:21:55 PM:   
8:21:55 PM: 
8:21:55 PM: not finished Merge worker state - 0.077s
8:21:55 PM: 
8:21:55 PM: ────────────────────────────────────────────────────────────────
8:21:55 PM:   "build.command" failed                                        
8:21:55 PM: ────────────────────────────────────────────────────────────────

djben · March 1, 2022, 5:02pm

It looks related to parralel image processing introduced in recent versions of Gatsby.

I’ve forced my Gatsby version to =v4.1.3 in package.json and my site builds (chances are other 4.x versions work, but definitely NOT 4.8 or 4.9), though still with a reduced set of static content because of the original issue here (memory usage is still higher than it should be with v4 in general).

hillary · March 1, 2022, 8:56pm

Hey there folks!

@homearanya, can you share a link to your Netlify site as well as your project repo? @djben, thanks for sharing those insights.

Massick · March 1, 2022, 11:16pm

I don’t know if this is the same issue, but I’m getting error UNHANDLED EXCEPTION write EPIPE everytime I try to deploy my Gatsby site. The first time I wasn’t making any “heavy change”, just some links updated.
My Netlify site is https://massick.netlify.app/ and my project repo is GitHub - M4ss1ck/massick-portfolio: My personal portfolio and Blog. My concern is that a few days back it was working fine with gatsby@4.7.1, and today it didn’t.
gatsby info results:

System:
    OS: Windows 10 10.0.19042
    CPU: (4) x64 Intel(R) Core(TM) i3-6100 CPU @ 3.70GHz
  Binaries:
    Node: 16.13.0 - C:\Program Files\nodejs\node.EXE
    npm: 8.3.0 - C:\Program Files\nodejs\npm.CMD
  Languages:
    Python: 3.10.1
  Browsers:
    Edge: Spartan (44.19041.423.0)
  npmPackages:
    gatsby: ^4.9.0 => 4.9.0
    gatsby-plugin-dark-mode: ^1.1.2 => 1.1.2
    gatsby-plugin-gatsby-cloud: ^4.6.0 => 4.7.0
    gatsby-plugin-google-analytics: ^4.6.0 => 4.7.0
    gatsby-plugin-image: ^2.6.0 => 2.7.0
    gatsby-plugin-manifest: ^4.6.0 => 4.7.0
    gatsby-plugin-netlify: ^4.1.0 => 4.1.0
    gatsby-plugin-offline: ^5.6.0 => 5.7.0
    gatsby-plugin-pnpm: ^1.2.10 => 1.2.10
    gatsby-plugin-postcss: ^5.6.0 => 5.7.0
    gatsby-plugin-react-helmet: ^5.6.0 => 5.7.0
    gatsby-plugin-react-i18next: ^1.2.2 => 1.2.2
    gatsby-plugin-sharp: ^4.6.0 => 4.7.0
    gatsby-plugin-sitemap: ^5.6.0 => 5.7.0
    gatsby-plugin-svgr-loader: ^0.1.0 => 0.1.0
    gatsby-plugin-webpack-bundle-analyser-v2: ^1.1.26 => 1.1.26
    gatsby-remark-copy-linked-files: ^5.6.0 => 5.7.0
    gatsby-remark-images: ^6.6.0 => 6.7.0
    gatsby-remark-prismjs: ^6.6.0 => 6.7.0
    gatsby-remark-responsive-iframe: ^5.6.0 => 5.7.0
    gatsby-remark-smartypants: ^5.6.0 => 5.7.0
    gatsby-source-filesystem: ^4.6.0 => 4.7.0
    gatsby-transformer-remark: ^5.6.0 => 5.7.0
    gatsby-transformer-sharp: ^4.6.0 => 4.7.0

Massick · March 1, 2022, 11:29pm

UPDATE: I just set GATSBY_CONCURRENT_DOWNLOAD to 16 as in this issue and my site is live

hillary · March 2, 2022, 3:45pm

Thanks for coming back and letting us know, @Massick! Happy building

homearanya · March 4, 2022, 8:29am

Hi Hillary,

Sorry for the late reply. Somehow your reply ended up on my spam folder

This is the netlify site: https://app.netlify.com/sites/chattermill/overview

The project repo is private but we can give you access if you need it. I’d need your email address.

Kind regards,

Carlos

homearanya · March 4, 2022, 9:42am

That did not work for me on:

GATSBY_CONCURRENT_DOWNLOAD = "16"

  npmPackages:
    gatsby: ^4.1.3 => 4.9.1
    gatsby-plugin-analytics: ^0.2.0 => 0.2.0
    gatsby-plugin-anchor-links: ^1.2.1 => 1.2.1
    gatsby-plugin-canonical-urls: ^4.9.0 => 4.9.0
    gatsby-plugin-force-trailing-slashes: ^1.0.5 => 1.0.6
    gatsby-plugin-google-analytics: ^4.9.0 => 4.9.0
    gatsby-plugin-google-tagmanager: ^4.9.0 => 4.9.0
    gatsby-plugin-hubspot: ^2.0.0 => 2.0.0
    gatsby-plugin-image: ^2.9.0 => 2.9.0
    gatsby-plugin-intercom-spa: ^0.2.0 => 0.2.0
    gatsby-plugin-linkedin-insight: ^1.0.1 => 1.0.1
    gatsby-plugin-manifest: 4.8.1 => 4.8.1
    gatsby-plugin-netlify: ^4.1.0 => 4.1.0
    gatsby-plugin-nprogress: 4.8.0 => 4.8.0
    gatsby-plugin-offline: 5.8.1 => 5.8.1
    gatsby-plugin-preconnect: ^1.3.0 => 1.3.0
    gatsby-plugin-react-helmet: ^5.9.0 => 5.9.0
    gatsby-plugin-react-svg: ^3.1.0 => 3.1.0
    gatsby-plugin-remove-serviceworker: ^1.0.0 => 1.0.0
    gatsby-plugin-robots-txt: ^1.7.0 => 1.7.0
    gatsby-plugin-sass: ^5.9.0 => 5.9.0
    gatsby-plugin-segment-js: ^3.7.1 => 3.7.1
    gatsby-plugin-sharp: ^4.9.0 => 4.9.0
    gatsby-plugin-sitemap: ^5.9.0 => 5.9.0
    gatsby-plugin-styled-components: ^5.9.0 => 5.9.0
    gatsby-plugin-typegen: ^2.2.4 => 2.2.4
    gatsby-remark-copy-linked-files: ^5.9.0 => 5.9.0
    gatsby-remark-images: ^6.9.0 => 6.9.0
    gatsby-remark-images-contentful: ^5.9.0 => 5.9.0
    gatsby-source-contentful: ^7.6.1 => 7.7.1
    gatsby-source-filesystem: ^4.9.0 => 4.9.0
    gatsby-transformer-remark: ^5.9.0 => 5.9.0
    gatsby-transformer-sharp: ^4.9.0 => 4.9.0

Topic		Replies	Views
Gatsby 4 Build Issue Works Local but Fails on Netlify Support deployment , gatsby	2	1570	March 28, 2022
Gatsby builds locally but fails on deploy Support building , deployment , gatsby	3	1704	February 27, 2021
'gatsby build' works locally but not on Netlify Support deployment	19	2159	March 29, 2025
Local build was successful but build at Netlify is not working Support building , deployment , gatsby	2	2064	April 25, 2020
Build works locally but deploy fails when pushed to github Support deployment , netlify-newbie , gatsby	5	2038	March 1, 2023

Gatsby v4 works locally, but timed out on Netlify

Related topics