Gatsby v4 works locally, but timed out on Netlify

hey team,

I finally managed to migrate my current repo to Gatsby v4 and it compiles locally, but times out on Netlify. I‘ve tried a couple of suggestions I‘ve found on Google:

So far nothing works. Here‘s the Netlify deploy build log.

Is it because gastby v4 is not officially supported yet or I am missing something important? Any pointer or suggestion is greatly appreciated, thank you!

I’m also seeing this same issue. The build times out no the Building production JavaScript and CSS bundles step with this message:

success run queries in workers - 438.798s - 1795/1795 4.09/s
8:00:06 AM: error UNHANDLED EXCEPTION write EPIPE
8:00:06 AM: 
8:00:06 AM: 
8:00:06 AM:   Error: write EPIPE
8:00:06 AM:   
8:00:06 AM:   - child_process.js:841 ChildProcess.target._send
8:00:06 AM:     internal/child_process.js:841:20
8:00:06 AM:   
8:00:06 AM:   - child_process.js:712 ChildProcess.target.send
8:00:06 AM:     internal/child_process.js:712:19
8:00:06 AM:   
8:00:06 AM:   - index.js:290 WorkerPool.sendMessage
8:00:06 AM:     [site]/[gatsby-worker]/dist/index.js:290:19
8:00:06 AM:   
8:00:06 AM:   - worker-messaging.ts:22 
8:00:06 AM:     [site]/[gatsby]/src/utils/jobs/worker-messaging.ts:22:22
8:00:06 AM:   
8:00:06 AM: 
8:00:06 AM: not finished Merge worker state - 0.260s
8:00:06 AM: 

The Gatsby team fixed the issue in build #4.0.1. That version works for me locally but times out on Netlify:

seems like so many things are still not working with v4 especially the ones with community plugins :man_shrugging:

hi there @ajmalafif - sorry to hear that there are still some rough edges. Please do keep continuing to file issues - that is the best way for the team that works on these specific things to learn about bugs you are experiencing so they can work on them efficiently! thank you.

I’m happy to file an issue and logs. Where do you want it to be reported?

here please! if it seems like its an issue with our plugin:

I’m seeing this same ‘write EPIPE’ error, referenced here with Gatsby (they should probably close this issue, I only responded there as it was the first place I saw it):

You’ll see in my comment there, I validated a theory posted that this is a memory or timeout issue.

If I drastically reduce the static content of my site, Netlify deploys it just fine.

Can someone at Netlify try bumping up my build timeout and/or memory limit?

My Netlify instance title is amazing-wiles-1918a6 :slight_smile:

Unfortunately, memory limit cannot be changed. We can bump the build time, but since it’s not a build time issue, I don’t see the benefit in doing this.

This might shed more info:

Maybe you can just bump my build time for now? You can see in my deploy history I have successful vs. unsuccessful deploys with V4 – maybe there is a log to check for memory usage differences.

I’ll also work on trying to pull down the Netlify docker image and checking memory usage myself.

Sure, the time limit has been bumped to 30 mins for now.

Unfortunately, we don’t log the build memory usage yet, so we don’t have that data.

So I went through the process for the first time of using Netlify’s build-image as @hrishikesh suggested.

It took a little bit of work and I ran into a bug where the submodule for all of my static assets wasn’t pulling down properly (Submodules do not get copied on cloning to container · Issue #444 · netlify/build-image · GitHub), but upon manually copying into assets to the build image volume in the right folder, I was able to start building my app.

With the default memory limit of 3GB, I seem to get the same write EPIPE error and my build fails (good, I reproduced it!).

I then bumped by Docker memory limit to 4GB and… the build succeeds.

I guess the question now is deciding how I move forward if the 3GB limit for Netlify builds cannot be changed :pensive:

Did the memory limit get reduced below 3GB recently, or were there any other changes to the build environments recently?

I have not been able to run a deploy of my app since February 14th, despite reducing it’s static content to the bare minimum :frowning:

2 Likes

Definitely something has changed on Netlify since Feb 14th. I had to downgrade to Gatsby v3. Same old issue on Gatsby v4

8:20:45 PM: success run queries in workers - 42.674s - 478/478 11.20/s
8:21:55 PM: success Running gatsby-plugin-sharp.IMAGE_PROCESSING jobs - 111.872s - 270/270 2.41/s
8:21:55 PM: error UNHANDLED EXCEPTION write EPIPE
8:21:55 PM: 
8:21:55 PM: 
8:21:55 PM:   Error: write EPIPE
8:21:55 PM:   
8:21:55 PM:   - child_process.js:841 ChildProcess.target._send
8:21:55 PM:     internal/child_process.js:841:20
8:21:55 PM:   
8:21:55 PM:   - child_process.js:712 ChildProcess.target.send
8:21:55 PM:     internal/child_process.js:712:19
8:21:55 PM:   
8:21:55 PM:   - index.js:291 WorkerPool.sendMessage
8:21:55 PM:     [repo]/[gatsby-worker]/dist/index.js:291:19
8:21:55 PM:   
8:21:55 PM:   - worker-messaging.ts:22 
8:21:55 PM:     [repo]/[gatsby]/src/utils/jobs/worker-messaging.ts:22:22
8:21:55 PM:   
8:21:55 PM: 
8:21:55 PM: not finished Merge worker state - 0.077s
8:21:55 PM: ​
8:21:55 PM: ────────────────────────────────────────────────────────────────
8:21:55 PM:   "build.command" failed                                        
8:21:55 PM: ────────────────────────────────────────────────────────────────
3 Likes

It looks related to parralel image processing introduced in recent versions of Gatsby.

I’ve forced my Gatsby version to =v4.1.3 in package.json and my site builds (chances are other 4.x versions work, but definitely NOT 4.8 or 4.9), though still with a reduced set of static content because of the original issue here (memory usage is still higher than it should be with v4 in general).

Hey there folks!

@homearanya, can you share a link to your Netlify site as well as your project repo? @djben, thanks for sharing those insights.

I don’t know if this is the same issue, but I’m getting error UNHANDLED EXCEPTION write EPIPE everytime I try to deploy my Gatsby site. The first time I wasn’t making any “heavy change”, just some links updated.
My Netlify site is https://massick.netlify.app/ and my project repo is GitHub - M4ss1ck/massick-portfolio. My concern is that a few days back it was working fine with gatsby@4.7.1, and today it didn’t.
gatsby info results:

System:
    OS: Windows 10 10.0.19042
    CPU: (4) x64 Intel(R) Core(TM) i3-6100 CPU @ 3.70GHz
  Binaries:
    Node: 16.13.0 - C:\Program Files\nodejs\node.EXE
    npm: 8.3.0 - C:\Program Files\nodejs\npm.CMD
  Languages:
    Python: 3.10.1
  Browsers:
    Edge: Spartan (44.19041.423.0)
  npmPackages:
    gatsby: ^4.9.0 => 4.9.0
    gatsby-plugin-dark-mode: ^1.1.2 => 1.1.2
    gatsby-plugin-gatsby-cloud: ^4.6.0 => 4.7.0
    gatsby-plugin-google-analytics: ^4.6.0 => 4.7.0
    gatsby-plugin-image: ^2.6.0 => 2.7.0
    gatsby-plugin-manifest: ^4.6.0 => 4.7.0
    gatsby-plugin-netlify: ^4.1.0 => 4.1.0
    gatsby-plugin-offline: ^5.6.0 => 5.7.0
    gatsby-plugin-pnpm: ^1.2.10 => 1.2.10
    gatsby-plugin-postcss: ^5.6.0 => 5.7.0
    gatsby-plugin-react-helmet: ^5.6.0 => 5.7.0
    gatsby-plugin-react-i18next: ^1.2.2 => 1.2.2
    gatsby-plugin-sharp: ^4.6.0 => 4.7.0
    gatsby-plugin-sitemap: ^5.6.0 => 5.7.0
    gatsby-plugin-svgr-loader: ^0.1.0 => 0.1.0
    gatsby-plugin-webpack-bundle-analyser-v2: ^1.1.26 => 1.1.26
    gatsby-remark-copy-linked-files: ^5.6.0 => 5.7.0
    gatsby-remark-images: ^6.6.0 => 6.7.0
    gatsby-remark-prismjs: ^6.6.0 => 6.7.0
    gatsby-remark-responsive-iframe: ^5.6.0 => 5.7.0
    gatsby-remark-smartypants: ^5.6.0 => 5.7.0
    gatsby-source-filesystem: ^4.6.0 => 4.7.0
    gatsby-transformer-remark: ^5.6.0 => 5.7.0
    gatsby-transformer-sharp: ^4.6.0 => 4.7.0

UPDATE: I just set GATSBY_CONCURRENT_DOWNLOAD to 16 as in this issue and my site is live

4 Likes

Thanks for coming back and letting us know, @Massick! Happy building :rocket:

1 Like

Hi Hillary,

Sorry for the late reply. Somehow your reply ended up on my spam folder

This is the netlify site: https://app.netlify.com/sites/chattermill/overview

The project repo is private but we can give you access if you need it. I’d need your email address.

Kind regards,

Carlos

That did not work for me :frowning: on:

GATSBY_CONCURRENT_DOWNLOAD = "16"
  npmPackages:
    gatsby: ^4.1.3 => 4.9.1
    gatsby-plugin-analytics: ^0.2.0 => 0.2.0
    gatsby-plugin-anchor-links: ^1.2.1 => 1.2.1
    gatsby-plugin-canonical-urls: ^4.9.0 => 4.9.0
    gatsby-plugin-force-trailing-slashes: ^1.0.5 => 1.0.6
    gatsby-plugin-google-analytics: ^4.9.0 => 4.9.0
    gatsby-plugin-google-tagmanager: ^4.9.0 => 4.9.0
    gatsby-plugin-hubspot: ^2.0.0 => 2.0.0
    gatsby-plugin-image: ^2.9.0 => 2.9.0
    gatsby-plugin-intercom-spa: ^0.2.0 => 0.2.0
    gatsby-plugin-linkedin-insight: ^1.0.1 => 1.0.1
    gatsby-plugin-manifest: 4.8.1 => 4.8.1
    gatsby-plugin-netlify: ^4.1.0 => 4.1.0
    gatsby-plugin-nprogress: 4.8.0 => 4.8.0
    gatsby-plugin-offline: 5.8.1 => 5.8.1
    gatsby-plugin-preconnect: ^1.3.0 => 1.3.0
    gatsby-plugin-react-helmet: ^5.9.0 => 5.9.0
    gatsby-plugin-react-svg: ^3.1.0 => 3.1.0
    gatsby-plugin-remove-serviceworker: ^1.0.0 => 1.0.0
    gatsby-plugin-robots-txt: ^1.7.0 => 1.7.0
    gatsby-plugin-sass: ^5.9.0 => 5.9.0
    gatsby-plugin-segment-js: ^3.7.1 => 3.7.1
    gatsby-plugin-sharp: ^4.9.0 => 4.9.0
    gatsby-plugin-sitemap: ^5.9.0 => 5.9.0
    gatsby-plugin-styled-components: ^5.9.0 => 5.9.0
    gatsby-plugin-typegen: ^2.2.4 => 2.2.4
    gatsby-remark-copy-linked-files: ^5.9.0 => 5.9.0
    gatsby-remark-images: ^6.9.0 => 6.9.0
    gatsby-remark-images-contentful: ^5.9.0 => 5.9.0
    gatsby-source-contentful: ^7.6.1 => 7.7.1
    gatsby-source-filesystem: ^4.9.0 => 4.9.0
    gatsby-transformer-remark: ^5.9.0 => 5.9.0
    gatsby-transformer-sharp: ^4.9.0 => 4.9.0