So this keeps happening for my new Gatsby site. For some reason everything hangs after sourcing nodes. Here is the latest deploy log, unfortunately I don’t really have much on my end to go off of and troubleshoot: Netlify App.
Any ideas?
howdy,
so the clues are in there:
7:38:49 AM: success source and transform nodes - 10.834s
8:07:22 AM: Build exceeded maximum allowed runtime
or, more specifically, in the time difference between these two timestamps.
Our regular maximum build time for a site is 15 minutes, and your build obvsly doesn’t complete within that timeframe.
I’m not exactly sure yet why not - you may have a lingering background process that never terminates and so we can’t complete the build.
More here:
The solution is to either a.) improve your build efficiency - this is where i would start.
Some more reading:
or b.) ask us to up your build time to 30 minutes. We can do this if there is a credit card associated with the account, so that we can automatically bill you for more build minutes, should you need them.
Thanks @perry it actually showed me that message after I posted. Thing is builds should probably be only taking 2-3 mins. Not sure what is causing the delays here, would you be able to pull anything up on your end?
@perry so I narrowed it down to some sort of issue with the part of my build process that fetches data from our CRM. Right now I am running a series of get requests to fetch all of the data we need to then source into Gatsby’s schema. It runs quickly on my system and the logs on Netlify seem to indicate that it successfully fetched and created the nodes. But then it just pauses with nothing further. If I remove this step the build is able to move forward. Is there a limit to the amount of get requests we can make during a build process?
Hi, @kmcaloon. I think this support guide might have more information about the root cause:
To summarize, some subprocess isn’t existing which then causes the build system to wait until it does. This doesn’t ever happen to the build is manually timed out after 15 minutes.
If this is the root cause, the solution is to make sure that subprocess exits and returns control to the build system.
If this doesn’t work or if there are other questions, please let us know.
Thanks @luke. I’ve been trying to debug this for a while. I think I’ve found the issue but really not sure what’s going on. It looks like the subprocess is related to a POST request I am making to get an access token to one of my APIs. I’ve tried everything I know to do but I’m just not able to fix this for my Netlify builds. I’m using axios and have tried using a custom https agent with a timeout and keepAlive: false, but nothing is working. I am sort of at a loss right now how to troubleshoot further. Any ideas?
You’re doing this during build, right? How are you making the call to that remote service, in more detail? What happens if you log it a bit more verbosely? (maybe something like: “about to connect to URL” "starting connection to URL at " "completed connection at " - that will make sure your URL is correct and show when the connection attempt was even if it doesn’t return - but it will also show if it DOES return in case the “stuckness” happens after).
I can tell you that we don’t block outgoing network connections (or responses thereto) during build, but if you connect and your remote end never returns an answer or something, that would manifest in the same way.
Right, that’s the confusing part. I am getting a response and everything seems to be logging as it should be. I am connecting via axios with pretty much every configuration I could imagine, including a custom https agent with keepAlive: false, but have also tried connecting via isomorphic-unfetch.
What seems to be throwing the issue is a custom login function interacting with a third-party API using OAuth for authentication. I am making a POST request with my refresh token to an endpoint that then successfully returns a fresh token as it should. Here is the response I am getting on Netlify (secured info hidden):
7:27:52 AM: {
7:27:52 AM: status: 200,
7:27:52 AM: statusText: '',
7:27:52 AM: headers: {
7:27:52 AM: server: 'ZGS',
7:27:52 AM: date: 'Tue, 11 Aug 2020 11:27:52 GMT',
7:27:52 AM: 'content-type': 'application/json;charset=UTF-8',
7:27:52 AM: 'content-length': '169',
7:27:52 AM: connection: 'close',
7:27:52 AM: 'set-cookie': [
7:27:52 AM: '****; Path=/',
7:27:52 AM: 'iamcsr=****;path=/;SameSite=None;Secure;priority=high',
7:27:52 AM: '_zcsr_tmp=7****;path=/;SameSite=Strict;Secure;priority=high'
7:27:52 AM: ],
7:27:52 AM: 'x-content-type-options': 'nosniff',
7:27:52 AM: 'x-xss-protection': '1',
7:27:52 AM: 'x-frame-options': 'SAMEORIGIN',
7:27:52 AM: 'strict-transport-security': 'max-age=63072000'
7:27:52 AM: },
7:27:52 AM: config: {
7:27:52 AM: url: 'https://accounts.zoho.com/oauth/v2/token',
7:27:52 AM: method: 'post',
7:27:52 AM: params: {
7:27:52 AM: refresh_token: '****',
7:27:52 AM: client_id: '****',
7:27:52 AM: client_secret: '****',
7:27:52 AM: grant_type: 'refresh_token'
7:27:52 AM: },
7:27:52 AM: headers: {
7:27:52 AM: Accept: 'application/json, text/plain, */*',
7:27:52 AM: 'Content-Type': 'application/x-www-form-urlencoded',
7:27:52 AM: 'User-Agent': 'axios/0.19.2'
7:27:52 AM: },
7:27:52 AM: transformRequest: [ [Function: transformRequest] ],
7:27:52 AM: transformResponse: [ [Function: transformResponse] ],
7:27:52 AM: timeout: 5000,
7:27:52 AM: adapter: [Function: httpAdapter],
7:27:52 AM: xsrfCookieName: 'XSRF-TOKEN',
7:27:52 AM: xsrfHeaderName: 'X-XSRF-TOKEN',
7:27:52 AM: maxContentLength: -1,
7:27:52 AM: validateStatus: [Function: validateStatus],
7:27:52 AM: httpsAgent: Agent {
7:27:52 AM: _events: [Object: null prototype],
7:27:52 AM: _eventsCount: 2,
7:27:52 AM: _maxListeners: undefined,
7:27:52 AM: defaultPort: 443,
7:27:52 AM: protocol: 'https:',
7:27:52 AM: options: [Object],
7:27:52 AM: requests: {},
7:27:52 AM: sockets: [Object],
7:27:52 AM: freeSockets: {},
7:27:52 AM: keepAliveMsecs: 1000,
7:27:52 AM: keepAlive: false,
7:27:52 AM: maxSockets: Infinity,
7:27:52 AM: maxFreeSockets: 256,
7:27:52 AM: maxCachedSessions: 100,
7:27:52 AM: _sessionCache: [Object],
7:27:52 AM: [Symbol(kCapture)]: false
7:27:52 AM: },
7:27:52 AM: data: undefined
7:27:52 AM: },
7:27:52 AM: request: ClientRequest {
7:27:52 AM: _events: [Object: null prototype] {
7:27:52 AM: socket: [Array],
7:27:52 AM: abort: [Function],
7:27:52 AM: aborted: [Function],
7:27:52 AM: error: [Function],
7:27:52 AM: timeout: [Array],
7:27:52 AM: prefinish: [Function: requestOnPrefinish]
7:27:52 AM: },
7:27:52 AM: _eventsCount: 6,
7:27:52 AM: _maxListeners: undefined,
7:27:52 AM: outputData: [],
7:27:52 AM: outputSize: 0,
7:27:52 AM: writable: true,
7:27:52 AM: _last: true,
7:27:52 AM: chunkedEncoding: false,
7:27:52 AM: shouldKeepAlive: false,
7:27:52 AM: useChunkedEncodingByDefault: true,
7:27:52 AM: sendDate: false,
7:27:52 AM: _removedConnection: false,
7:27:52 AM: _removedContLen: false,
7:27:52 AM: _removedTE: false,
7:27:52 AM: _contentLength: 0,
7:27:52 AM: _hasBody: true,
7:27:52 AM: _trailer: '',
7:27:52 AM: finished: true,
7:27:52 AM: _headerSent: true,
7:27:52 AM: socket: TLSSocket {
7:27:52 AM: _tlsOptions: [Object],
7:27:52 AM: _secureEstablished: true,
7:27:52 AM: _securePending: false,
7:27:52 AM: _newSessionPending: false,
7:27:52 AM: _controlReleased: true,
7:27:52 AM: secureConnecting: true,
7:27:52 AM: _SNICallback: null,
7:27:52 AM: servername: 'accounts.zoho.com',
7:27:52 AM: alpnProtocol: false,
7:27:52 AM: authorized: true,
7:27:52 AM: authorizationError: null,
7:27:52 AM: encrypted: true,
7:27:52 AM: _events: [Object: null prototype],
7:27:52 AM: _eventsCount: 10,
7:27:52 AM: connecting: false,
7:27:52 AM: _hadError: false,
7:27:52 AM: _parent: null,
7:27:52 AM: _host: 'accounts.zoho.com',
7:27:52 AM: _readableState: [ReadableState],
7:27:52 AM: readable: true,
7:27:52 AM: _maxListeners: undefined,
7:27:52 AM: _writableState: [WritableState],
7:27:52 AM: writable: false,
7:27:52 AM: allowHalfOpen: false,
7:27:52 AM: _sockname: null,
7:27:52 AM: _pendingData: null,
7:27:52 AM: _pendingEncoding: '',
7:27:52 AM: server: undefined,
7:27:52 AM: _server: null,
7:27:52 AM: ssl: [TLSWrap],
7:27:52 AM: _requestCert: true,
7:27:52 AM: _rejectUnauthorized: true,
7:27:52 AM: parser: null,
7:27:52 AM: _httpMessage: [Circular],
7:27:52 AM: timeout: 5000,
7:27:52 AM: [Symbol(res)]: [TLSWrap],
7:27:52 AM: [Symbol(verified)]: true,
7:27:52 AM: [Symbol(pendingSession)]: null,
7:27:52 AM: [Symbol(asyncId)]: 5510,
7:27:52 AM: [Symbol(kHandle)]: [TLSWrap],
7:27:52 AM: [Symbol(kSetNoDelay)]: false,
7:27:52 AM: [Symbol(lastWriteQueueSize)]: 0,
7:27:52 AM: [Symbol(timeout)]: Timeout {
7:27:52 AM: _idleTimeout: 5000,
7:27:52 AM: _idlePrev: [TimersList],
7:27:52 AM: _idleNext: [TimersList],
7:27:52 AM: _idleStart: 6483,
7:27:52 AM: _onTimeout: [Function: bound ],
7:27:52 AM: _timerArgs: undefined,
7:27:52 AM: _repeat: null,
7:27:52 AM: _destroyed: false,
7:27:52 AM: [Symbol(refed)]: false,
7:27:52 AM: [Symbol(asyncId)]: 5520,
7:27:52 AM: [Symbol(triggerId)]: 5518
7:27:52 AM: },
7:27:52 AM: [Symbol(kBuffer)]: null,
7:27:52 AM: [Symbol(kBufferCb)]: null,
7:27:52 AM: [Symbol(kBufferGen)]: null,
7:27:52 AM: [Symbol(kCapture)]: false,
7:27:52 AM: [Symbol(kBytesRead)]: 0,
7:27:52 AM: [Symbol(kBytesWritten)]: 0,
7:27:52 AM: [Symbol(connect-options)]: [Object]
7:27:52 AM: },
7:27:52 AM: connection: TLSSocket {
7:27:52 AM: _tlsOptions: [Object],
7:27:52 AM: _secureEstablished: true,
7:27:52 AM: _securePending: false,
7:27:52 AM: _newSessionPending: false,
7:27:52 AM: _controlReleased: true,
7:27:52 AM: secureConnecting: true,
7:27:52 AM: _SNICallback: null,
7:27:52 AM: servername: 'accounts.zoho.com',
7:27:52 AM: alpnProtocol: false,
7:27:52 AM: authorized: true,
7:27:52 AM: authorizationError: null,
7:27:52 AM: encrypted: true,
7:27:52 AM: _events: [Object: null prototype],
7:27:52 AM: _eventsCount: 10,
7:27:52 AM: connecting: false,
7:27:52 AM: _hadError: false,
7:27:52 AM: _parent: null,
7:27:52 AM: _host: 'accounts.zoho.com',
7:27:52 AM: _readableState: [ReadableState],
7:27:52 AM: readable: true,
7:27:52 AM: _maxListeners: undefined,
7:27:52 AM: _writableState: [WritableState],
7:27:52 AM: writable: false,
7:27:52 AM: allowHalfOpen: false,
7:27:52 AM: _sockname: null,
7:27:52 AM: _pendingData: null,
7:27:52 AM: _pendingEncoding: '',
7:27:52 AM: server: undefined,
7:27:52 AM: _server: null,
7:27:52 AM: ssl: [TLSWrap],
7:27:52 AM: _requestCert: true,
7:27:52 AM: _rejectUnauthorized: true,
7:27:52 AM: parser: null,
7:27:52 AM: _httpMessage: [Circular],
7:27:52 AM: timeout: 5000,
7:27:52 AM: [Symbol(res)]: [TLSWrap],
7:27:52 AM: [Symbol(verified)]: true,
7:27:52 AM: [Symbol(pendingSession)]: null,
7:27:52 AM: [Symbol(asyncId)]: 5510,
7:27:52 AM: [Symbol(kHandle)]: [TLSWrap],
7:27:52 AM: [Symbol(kSetNoDelay)]: false,
7:27:52 AM: [Symbol(lastWriteQueueSize)]: 0,
7:27:52 AM: [Symbol(timeout)]: Timeout {
7:27:52 AM: _idleTimeout: 5000,
7:27:52 AM: _idlePrev: [TimersList],
7:27:52 AM: _idleNext: [TimersList],
7:27:52 AM: _idleStart: 6483,
7:27:52 AM: _onTimeout: [Function: bound ],
7:27:52 AM: _timerArgs: undefined,
7:27:52 AM: _repeat: null,
7:27:52 AM: _destroyed: false,
7:27:52 AM: [Symbol(refed)]: false,
7:27:52 AM: [Symbol(asyncId)]: 5520,
7:27:52 AM: [Symbol(triggerId)]: 5518
7:27:52 AM: },
7:27:52 AM: [Symbol(kBuffer)]: null,
7:27:52 AM: [Symbol(kBufferCb)]: null,
7:27:52 AM: [Symbol(kBufferGen)]: null,
7:27:52 AM: [Symbol(kCapture)]: false,
7:27:52 AM: [Symbol(kBytesRead)]: 0,
7:27:52 AM: [Symbol(kBytesWritten)]: 0,
7:27:52 AM: [Symbol(connect-options)]: [Object]
7:27:52 AM: },
7:27:52 AM: _header: 'POST /oauth/v2/token?refresh_token=****&client_id=****&client_secret=****&grant_type=refresh_token HTTP/1.1\r\n' +
7:27:52 AM: 'Accept: application/json, text/plain, */*\r\n' +
7:27:52 AM: 'Content-Type: application/x-www-form-urlencoded\r\n' +
7:27:52 AM: 'User-Agent: axios/0.19.2\r\n' +
7:27:52 AM: 'Host: accounts.zoho.com\r\n' +
7:27:52 AM: 'Connection: close\r\n' +
7:27:52 AM: 'Content-Length: 0\r\n' +
7:27:52 AM: '\r\n',
7:27:52 AM: _onPendingData: [Function: noopPendingOutput],
7:27:52 AM: agent: Agent {
7:27:52 AM: _events: [Object: null prototype],
7:27:52 AM: _eventsCount: 2,
7:27:52 AM: _maxListeners: undefined,
7:27:52 AM: defaultPort: 443,
7:27:52 AM: protocol: 'https:',
7:27:52 AM: options: [Object],
7:27:52 AM: requests: {},
7:27:52 AM: sockets: [Object],
7:27:52 AM: freeSockets: {},
7:27:52 AM: keepAliveMsecs: 1000,
7:27:52 AM: keepAlive: false,
7:27:52 AM: maxSockets: Infinity,
7:27:52 AM: maxFreeSockets: 256,
7:27:52 AM: maxCachedSessions: 100,
7:27:52 AM: _sessionCache: [Object],
7:27:52 AM: [Symbol(kCapture)]: false
7:27:52 AM: },
7:27:52 AM: socketPath: undefined,
7:27:52 AM: method: 'POST',
7:27:52 AM: insecureHTTPParser: undefined,
7:27:52 AM: path: '/oauth/v2/token?refresh_token=****&client_id=****&client_secret=****&grant_type=refresh_token',
7:27:52 AM: _ended: true,
7:27:52 AM: res: IncomingMessage {
7:27:52 AM: _readableState: [ReadableState],
7:27:52 AM: readable: false,
7:27:52 AM: _events: [Object: null prototype],
7:27:52 AM: _eventsCount: 3,
7:27:52 AM: _maxListeners: undefined,
7:27:52 AM: socket: [TLSSocket],
7:27:52 AM: connection: [TLSSocket],
7:27:52 AM: httpVersionMajor: 1,
7:27:52 AM: httpVersionMinor: 1,
7:27:52 AM: httpVersion: '1.1',
7:27:52 AM: complete: true,
7:27:52 AM: headers: [Object],
7:27:52 AM: rawHeaders: [Array],
7:27:52 AM: trailers: {},
7:27:52 AM: rawTrailers: [],
7:27:52 AM: aborted: false,
7:27:52 AM: upgrade: false,
7:27:52 AM: url: '',
7:27:52 AM: method: null,
7:27:52 AM: statusCode: 200,
7:27:52 AM: statusMessage: '',
7:27:52 AM: client: [TLSSocket],
7:27:52 AM: _consuming: false,
7:27:52 AM: _dumped: false,
7:27:52 AM: req: [Circular],
7:27:52 AM: responseUrl: 'https://accounts.zoho.com/oauth/v2/token?refresh_token=****&client_id=****&client_secret=****&grant_type=refresh_token',
7:27:52 AM: redirects: [],
7:27:52 AM: [Symbol(kCapture)]: false
7:27:52 AM: },
7:27:52 AM: aborted: false,
7:27:52 AM: timeoutCb: [Function: emitRequestTimeout],
7:27:52 AM: upgradeOrConnect: false,
7:27:52 AM: parser: null,
7:27:52 AM: maxHeadersCount: null,
7:27:52 AM: reusedSocket: false,
7:27:52 AM: _redirectable: Writable {
7:27:52 AM: _writableState: [WritableState],
7:27:52 AM: writable: true,
7:27:52 AM: _events: [Object: null prototype],
7:27:52 AM: _eventsCount: 2,
7:27:52 AM: _maxListeners: undefined,
7:27:52 AM: _options: [Object],
7:27:52 AM: _redirectCount: 0,
7:27:52 AM: _redirects: [],
7:27:52 AM: _requestBodyLength: 0,
7:27:52 AM: _requestBodyBuffers: [],
7:27:52 AM: _onNativeResponse: [Function],
7:27:52 AM: _currentRequest: [Circular],
7:27:52 AM: _currentUrl: 'https://accounts.zoho.com/oauth/v2/token?refresh_token=****&client_id=****&client_secret=****&grant_type=refresh_token',
7:27:52 AM: [Symbol(kCapture)]: false
7:27:52 AM: },
7:27:52 AM: [Symbol(kCapture)]: false,
7:27:52 AM: [Symbol(kNeedDrain)]: false,
7:27:52 AM: [Symbol(corked)]: 0,
7:27:52 AM: [Symbol(kOutHeaders)]: [Object: null prototype] {
7:27:52 AM: accept: [Array],
7:27:52 AM: 'content-type': [Array],
7:27:52 AM: 'user-agent': [Array],
7:27:52 AM: host: [Array]
7:27:52 AM: }
7:27:52 AM: },
7:27:52 AM: data: {
7:27:52 AM: access_token: '****',
7:27:52 AM: api_domain: 'https://www.zohoapis.com',
7:27:52 AM: token_type: 'Bearer',
7:27:52 AM: expires_in: 3600
7:27:52 AM: }
7:27:52 AM: }
I think we’ve wandered well past where we can still provide tech support - you can see the problem and reproduce it so hopefully that will be enough to work with zoho for them to advise on a fix in your code 
You might try reproducing locally to confirm, using this methodology: GitHub - netlify/build-image: This is the build image used for running automated builds - so you can gather error logs and retry auth requests.
Hi All,
I’m having this exact same issue.
Here is what I can add to this.
-
If I deploy using a webhook or via the Netlify interface, the build process will hang in exactly the same manner described by @kmcaloon.
-
However, if I deploy via a new commit, the build will complete as normal. Every time.
Is there a different process that Netlify uses between these two types of deployments?
False alarm. Builds will fail even from a commit. That seemed like a long shot anyway.
It will randomly succeed or mysteriously hang then time out for no discernable reason.
Me again.
100% it’s a file download that never finishes writing or errors out and doesn’t close.
I added a timeout and close events to errors. Builds are going ok.
But builds working OK is only half the issue. Build work ok everywhere, but only sometimes on Netlify.
Hi Luke,
Here is the id 609a90ee41e62f18bce976b1. You can see it just stops.
I was able to track it down to a timeout error not being handled. I have no idea why it does not timeout locally, ever.
An easy thing to overlook (guilty) when writing files with Node.JS, when an error occurs it won’t autoclose and the ‘finish’ will not fire. This would cause the promise chain to wait for the previous one to resolve, which it never would. It just sits there.
I added a custom timeout, error and retry for files that will close. It seems to be working fine now.
It’s not a Netlify issue (at least I don’t think it is).
1 Like
Hi, @FKAInteractive. Thank you for sharing the solution you discovered! This is incredibly helpful for people searching these forums for answers in the future.