We have about 40 Gatsby websites (using Sanity headless CMS) that are all essentially identical and hosted on Netlify. This morning I noticed that the contact forms on some of them were no longer working and returning 502 responses. These forms are all routing to a serverless Netlify function and have been working fine for many months.
Upon digging, I noticed a few things:
- The sites that were working had their builds triggered from within the Netlify dashboard or had not been built in many weeks
- The sites that were not working had their most recent build triggered via the Netlify build widget within Sanity and were built within the last few weeks.
Upon reviewing the differences between the two builds, I primarily noticed 2 things:
- @netlify/plugin-gatsby was loading on the non-working version
- .netlify/functions-internal were being loaded on the non-working version
From there, I reviewed the NPM documentation for @netlify/plugin-gatsby and found an environment variable called NETLIFY_SKIP_GATSBY_FUNCTIONS to disable Gatsby functions. Since i’m not using any Gatsby functions, I set this variable to false team wide and rebuilt using the widget which restored functionality of the contact forms across all of the sites.
Although I have found this solution, I was hoping to understand a few things to try and avoid this type of outage in the future. Since the builds were completing successfully in this case it could potentially go unnoticed for a period of time which is catastrophic for us.
- Why is the plugin being injected into the builds if we do not have the plugin in our projects packages?
- Did something change on Netlify’s end to begin injecting this plugin recently? If so was there a notification that we missed?
- Any idea why it was not being injected when the builds were started from within the Netlify portal but was injected using the Sanity widget to initiate the builds?
Any additional context or details that you have surrounding this is appreciated. Again my goal is to prevent outages like this going forward and try to uncover what caused this seemingly out of the blue.
Thanks in advance!