Deployment Failure with "Error Uploading Blobs"

Context:
We have been successfully deploying updates to our site for months. Recently, we encountered deployment failures when pushing updates to our dev branch. The issue seems to be related to “Error uploading blobs,” and we are unsure how to resolve this as we have not configured blob management before.

Site Information:

  • Site name: dev-misinfo-dashboard.netlify.app
  • DNS issues: None, we use the Netlify URL provided.

Build problems: from the deploy log:

12:38:01 PM: Uploading 24 blobs to deploy store...
12:38:01 PM: Error uploading blobs to deploy store: Netlify Blobs has generated an internal error (301 status code)
12:38:01 PM: ​
12:38:01 PM: Internal error during "Uploading blobs"                       
12:38:01 PM: ────────────────────────────────────────────────────────────────
12:38:01 PM: ​
12:38:01 PM:   Error message
12:38:01 PM:   Error: Failed while uploading blobs to deploy store
12:38:01 PM: ​
12:38:01 PM:   Error location
12:38:01 PM:   During Uploading blobs
12:38:01 PM:       at coreStep (file:///opt/buildhome/node-deps/node_modules/@netlify/build/lib/plugins_core/blobs_upload/index.js:60:15)
12:38:01 PM:       at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
12:38:01 PM:       at async fireCoreStep (file:///opt/buildhome/node-deps/node_modules/@netlify/build/lib/steps/core_step.js:11:98)
12:38:01 PM:       at async tFireStep (file:///opt/buildhome/node-deps/node_modules/@netlify/build/lib/time/main.js:18:63)
12:38:01 PM:       at async file:///opt/buildhome/node-deps/node_modules/@netlify/build/lib/steps/run_step.js:69:283
12:38:01 PM:       at async pReduce.index (file:///opt/buildhome/node-deps/node_modules/@netlify/build/lib/steps/run_steps.js:12:349)
12:38:01 PM:       at async Promise.all (index 0)
12:38:01 PM:       at async next (file:///opt/buildhome/node-deps/node_modules/p-reduce/index.js:15:44)
12:38:01 PM: ​
12:38:01 PM:   Resolved config
12:38:01 PM:   build:
12:38:01 PM:     command: yarn run build
12:38:01 PM:     commandOrigin: config
12:38:01 PM:     environment:
12:38:01 PM:       - NEXT_FORCE_EDGE_IMAGES
12:38:01 PM:       - NEXT_PUBLIC_FIREBASE_API_KEY
12:38:01 PM:       - NEXT_PUBLIC_FIREBASE_APPCHECK_DEBUG_TOKEN
12:38:01 PM:       - NEXT_PUBLIC_FIREBASE_APP_ID
12:38:01 PM:       - NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN
12:38:01 PM:       - NEXT_PUBLIC_FIREBASE_MEASUREMENT_ID
12:38:01 PM:       - NEXT_PUBLIC_FIREBASE_MESSAGING_SENDER_ID
12:38:01 PM:       - NEXT_PUBLIC_FIREBASE_PROJECT_ID
12:38:01 PM:       - NEXT_PUBLIC_FIREBASE_RECAPTCHA_ENTERPRISE_SITE_KEY
12:38:01 PM:       - NEXT_PUBLIC_FIREBASE_STORAGE_BUCKET
12:38:01 PM:     publish: /opt/build/repo/.next
12:38:01 PM:     publishOrigin: config
12:38:01 PM:   functionsDirectory: /opt/build/repo/netlify/functions
12:38:01 PM:   plugins:
12:38:01 PM:     - inputs: {}
12:38:01 PM:       origin: ui
12:38:01 PM:       package: '@netlify/plugin-nextjs'
12:38:01 PM:   redirects:
12:38:01 PM:     - force: true
12:38:01 PM:       from: /_next/image/*
      query:
        q: ':quality'
        url: ':url'
        w: ':width'
      status: 200
      to: /.netlify/images?url=:url&w=:width&q=:quality
    - from: /_next/image
      query:
        q: ':quality'
        url: ':url'
        w: ':width'
      status: 200
      to: /.netlify/images?url=:url&w=:width&q=:quality
    - from: /_ipx/*
      query:
        q: ':quality'
        url: ':url'
        w: ':width'
      status: 200
      to: /.netlify/images?url=:url&w=:width&q=:quality
  redirectsOrigin: inline
12:38:01 PM: Build failed due to an internal system error: Build script returned non-zero exit code: 4
12:38:02 PM: Failing build: Failed to build site
12:38:02 PM: Finished processing build request in 5m25.144s
12:38:02 PM: Failed during stage 'building site': Build script returned non-zero exit code: 4 (https://ntl.fyi/exit-code-4)

Troubleshooting Steps Taken:

  • Checked Ask Netlify: We were directed to the Netlify Blobs documentation, but the troubleshooting tips provided did not resolve our issue.

  • Added the next/image redirect snippet for Netlify Image CDN to our netlify.toml file as a possible fix.

Netlify UI settings:

  • Dependency management Node.js: 20.x
  • Production branch: dev
  • Runtime: Next.js
  • Base directory: /
  • Package directory: Not set
  • Build command: yarn run build
  • Publish directory: .next
  • Functions directory: /Users/lukey/Sites/misinfo/misinfo-dashboard/netlify/functions
  • Deploy log visibility: Logs are public
  • Build status: Active

Local project versions:

  • Yarn 1.22.11
  • Node v20.12.2
  • Next.js v14.2.3
  • React ^18.3.0
  • netlify-cli/17.23.8 darwin-arm64 node-v20.12.2

Request: We are seeking guidance on how to manage the blob configuration or any other steps we can take to resolve this deployment issue. Any help or pointers would be greatly appreciated.

Thanks again, Luke and the Dev Team

1 Like

We started experiencing the same problem at ~9am PST today (Friday May 24th). We had been successfully deploying for months without any deployment issue and the changes we pushed today were minor copy changes. If it helps we’re also using a nextjs app.

1 Like

We are not alone! Thanks @tommy for chiming in :slight_smile:

I forgot to post that our local dev (yarn dev) boots up fine and we are able to view and browse the site as always. Also the local build seems to run successfully. Running yarn run build outputs:

yarn run build
yarn run v1.22.11
$ next build
  ▲ Next.js 14.2.3
  - Environments: .env.local, .env

 ✓ Linting and checking validity of types    
   Creating an optimized production build ...
> [PWA] Compile server
> [PWA] Compile server
> [PWA] Compile client (static)
> [PWA] Auto register service worker with: /Users/lukey/Sites/misinfo/misinfo-dashboard/node_modules/next-pwa/register.js
> [PWA] Service worker: /Users/lukey/Sites/misinfo/misinfo-dashboard/public/sw.js
> [PWA]   url: /sw.js
> [PWA]   scope: /
> [PWA] Fallback to precache routes when fetch failed from cache or network:
> [PWA]   document (page): /_offline
 ✓ Compiled successfully
 ✓ Collecting page data    
 ✓ Generating static pages (32/32)
 ✓ Collecting build traces    
 ✓ Finalizing page optimization    

Route (pages)                              Size     First Load JS
┌ ○ /                                      300 B           402 kB
├   /_app                                  0 B             402 kB
├ ○ /_offline                              358 B           402 kB
├ ○ /404                                   182 B           402 kB
├ ƒ /api/hello                             0 B             402 kB
├ ● /dashboard                             163 kB          2.9 MB
├   └ css/e9b5c8e9ee5edd34.css             2.55 kB
├ ○ /dashboard/reports/[reportId]          5.66 kB         425 kB
├ ● /login                                 5.72 kB         411 kB
├ ● /report                                6.19 kB        2.74 MB
├ ○ /resetPassword                         1.19 kB         410 kB
├ ● /signup                                20.4 kB        2.72 MB
├   └ css/aa15fe3776697b1b.css             24.4 kB
├ ○ /testPage                              3.67 kB         406 kB
└ ○ /verifyEmail                           1.08 kB         406 kB
+ First Load JS shared by all              420 kB
  ├ chunks/framework-9620da855a94eb57.js   45.2 kB
  ├ chunks/main-d70a77781224ea50.js        35 kB
  ├ chunks/pages/_app-329f3866034b2a22.js  321 kB
  ├ css/42980fff1043b9c8.css               18.5 kB
  └ other shared chunks (total)            1.05 kB

○  (Static)   prerendered as static content
●  (SSG)      prerendered as static HTML (uses getStaticProps)
ƒ  (Dynamic)  server-rendered on demand

✨  Done in 19.54s.

Hope that helps.

I’m an engineering manager here at Netlify - we believe we have solved this build problem. If you are still experiencing issues, please let us know and sorry for the inconvenience.

2 Likes

Thanks @elmay-netlify1 I can confirm that our deploys are now passing :rocket:

Would it be possible to provide more information on the issue?

1 Like

This was a result of a change on our end that didn’t go as expected. It was rolled back as soon as errors were detected.

2 Likes

Thank you @elmay-netlify1 @hrishikesh for getting it fixed quickly. We were able to deploy without issue. Thanks again.