After deploying our site we seem to be consistently getting generic ‘This edge function has crashed’ errors on all pages, they hang for a while and finally return an error page. After a few minutes all pages work fine. Could that be a cold start issue at the edge? We are getting nothing at in the edge function log.
We can’t launch our new site because of it.
I’ve had the exact same experience a couple of weeks back. In my function, setting a cookie didn’t work, and the rest of the function relied on it. Not sure if that’s the case with yours, but if so, I’d start there.
Thanks for your reply. So in your case, you had the same issue with every page crashing for a few minutes and then working fine with absolutely no change on your end?
We’re losing our minds right now. We’re one week from release and this is blocking our entire launch. All the pressure is on me to get this right and questioning my vendor choice with Netlify. The log gives me nothing - if the page renders all edge functions log to the console just fine, if it doesn’t I get no error at all. Can’t be reproduced locally. And zero response from support. This is nuts.
As I’m typing the site is rendering and logging when viewed from Arizona. Where I am in Panama, I get this:
An unhandled error in the function code triggered the following message:
The deployment failed while serving the request.
Netlify internal ID: 01GESH129V5772Q5SJA8E8HEAG
Yes, that was just about the case. The function was working fine on localhost (ntl dev) but it wasn’t working in production. Sometimes, after switching urls, all of the sudden a page would show up. I was just testing some potential use case and abandoned the idea after a few hours. Guess edge functions are still in beta for a reason.
OK thanks. I see what you’re saying, but after making a big announcement 6 months ago, making the beta publicly available, with a ton of marketing content advertising the feature, and actually charging customers for its use, for Netlify to hide behind the ‘Beta’ label as an excuse for the entire service to be unreliable to the point of being unusable is just unacceptable. Right now we are completely screwed.
Can you share your function?
I could add you to the repo, we actually have a few edge functions, too much code to paste here, but honestly I’m afraid it would me a waste of your time. As I’m browsing now, behavior is completely erratic, the same exact page loads in 150ms a few times, then crashes a few times with a generic edge function error, then works again just fine, just by refreshing the page. No errors in log. No idea which function is crashing. Nothing to debug or trace. And errors can’t be reproduced locally using Netlify CLI.
Here’s the 11ty edge function, that’s the one that does the most work. We’ve alreadyoptimized the 11ty Edge generated file to be as small as possible, 45KB.
import * as djwt from 'https://deno.land/x/djwt@v2.4/mod.ts';
import { EleventyEdge, precompiledAppData } from "./_generated/eleventy-edge-app.js";
import * as KEYS from './lib/cryptoKeys.js';
import isPathExcluded from './lib/excluded-paths.js';
import * as cookie from 'https://deno.land/std@0.151.0/http/cookie.ts';
import userNavigation from './_generated/userNavigation.json' assert {type: 'json'};
const TOKEN_NAME = 'JWT';
const muxKeyId = Deno.env.get('MUX_SIGNING_KEY_ID');
function trim(str, ch) {
let start = 0,
end = str.length;
while(start < end && str[start] === ch)
++start;
while(end > start && str[end - 1] === ch)
--end;
return (start > 0 || end < str.length) ? str.substring(start, end) : str;
}
const variables = {
algoliaApplicationId: Deno.env.get('ALGOLIA_APP_ID'),
algoliaPublicKey: Deno.env.get('ALGOLIA_PUBLIC_KEY'),
magicPublicKey: Deno.env.get('MAGIC_PUBLIC_KEY'),
};
const icons = {
edit: `<svg aria-hidden="true" focusable="false" width="24" height="24" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M12.0002 19.9991H19.2C19.6416 19.9991 20 19.6242 20 19.1624C20 18.7005 19.6416 18.3256 19.2 18.3256H12.0002C11.5586 18.3256 11.2002 18.7005 11.2002 19.1624C11.2002 19.6242 11.5586 19.9991 12.0002 19.9991ZM15.0345 4.76477L5.03469 15.2239C4.93709 15.3251 4.86109 15.4582 4.82429 15.613L4.02431 18.9599C3.99311 19.0854 3.99071 19.2259 4.02431 19.3657C4.13151 19.8142 4.5659 20.0869 4.99469 19.9748L8.19463 19.1381C8.32822 19.1038 8.45862 19.0301 8.56662 18.918L18.5664 8.45893C19.0536 7.94936 19.2976 7.27914 19.2976 6.61227C19.2976 5.94539 19.0536 5.27517 18.5664 4.76561C18.0792 4.25604 17.4376 4 16.8001 4C16.1625 4 15.5217 4.2552 15.0345 4.76477ZM16.1657 5.9479C16.3409 5.76466 16.5689 5.67346 16.8001 5.67346C17.0313 5.67346 17.2593 5.76466 17.4344 5.9479C17.6096 6.13115 17.6968 6.36961 17.6968 6.61143C17.6968 6.85324 17.6096 7.09171 17.4344 7.27495L7.59144 17.5701L5.89947 18.0127L6.32266 16.243L16.1657 5.9479Z" fill="currentColor"/>
</svg>`,
arrow: `<svg aria-hidden="true" focusable="false" width="24" height="24" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M9.13393 20.6234L16.6335 12.9092C17.1222 12.4065 17.1222 11.5927 16.6335 11.0912L9.13393 3.37703C8.64521 2.87432 7.85401 2.87432 7.36654 3.37703C6.87907 3.87974 6.87782 4.69359 7.36654 5.19502L13.9824 12.0002L7.36654 18.8055C6.87782 19.3082 6.87782 20.122 7.36654 20.6234C7.85526 21.1249 8.64646 21.1262 9.13393 20.6234Z" fill="currentColor"/>
</svg>`
};
export default async (request, context) => {
const start = Date.now();
const url = new URL(request.url);
if (isPathExcluded(url.pathname)) return;
try {
const urlParams = [...url.searchParams.entries()].reduce((obj, entry) => {
obj[entry[0]] = entry[1];
return obj;
}, {});
const cookies = cookie.getCookies(request.headers);
urlParams.slug = trim(url.pathname, '/');
urlParams.pathname = url.pathname;
urlParams.queryString = url.search;
urlParams.referer = request.headers.get('referer');
const edge = new EleventyEdge('edge', {
request,
context,
precompiled: precompiledAppData
});
const token = context.cookies.get(TOKEN_NAME);
const jwtKey = token ? await KEYS.JWT() : undefined;
let user = token ? await djwt.verify(token, jwtKey) : undefined;
if (user) {
const key = await KEYS.INTERCOM();
const buff = new TextEncoder().encode(user.email);
const hashBuffer = await crypto.subtle.sign('HMAC', key, buff);
const hashArray = Array.from(new Uint8Array(hashBuffer)); // convert buffer to byte array
const intercomhash = hashArray.map((b) => b.toString(16).padStart(2, '0')).join(''); // convert bytes to hex string
const nameArray = (user.name || '').split(' ');
user = {
...user,
intercomhash,
firstName: nameArray.length ? nameArray[0] : '',
lastName: nameArray.length > 1 ? nameArray[nameArray.length - 1] : '',
};
}
const formDataToken = context.cookies.get('FORM_DATA_TOKEN');
const formData = formDataToken && (await djwt.verify(formDataToken, jwtKey));
edge.config(eleventyConfig => {
eleventyConfig.addGlobalData('url', urlParams);
eleventyConfig.addGlobalData('userNavigation', userNavigation);
eleventyConfig.addGlobalData('cookies', cookies);
eleventyConfig.addGlobalData('icons', icons);
eleventyConfig.addGlobalData('variables', variables);
if (user) eleventyConfig.addGlobalData('user', user)
if (formData) eleventyConfig.addGlobalData('formData', formData);
eleventyConfig.addFilter('urlEncode', value => encodeURIComponent(value));
eleventyConfig.addNunjucksAsyncShortcode("muxSig", async (playbackId, audience) => {
try{
const key = await KEYS.MUX();
const token = await djwt.create(
{
alg: 'RS256',
typ: 'JWT'
},
{
sub: playbackId,
aud: audience,
exp: djwt.getNumericDate(60 * 60 * 24 * 1), // 1 day
kid: muxKeyId
},
key
);
return token;
}catch(err){
context.log(err);
}
});
});
const resp = await edge.handleResponse();
context.log(`Processing time: ${(Date.now() - start)}ms`);
return resp;
} catch (e) {
context.log( "ERROR", { e });
return context.next(e);
}
};
@slegay, I’m sorry, but I’m going to have to ask you to calm down. As I’ve mentioned in your helpdesk thread, opening multiple issues, posting on various places, is not going to help! Your frustration is valid, but at the moment, by posting about the same issue on multiple places, you’re not achieving anything - your actual issue at hand is not getting resolved.
Apart from these threads, you have also left feedback in Netlify Docs. All of this has been noted, and you might feel better to know that Product Manager of Edge Functions has also looked over your feedback. So, this situation will change - sooner or later.
About this, let me give you some context:
Netlify had started working on Edge Handlers since 2020. We had spent a lot of engineering effort into making that, there were private betas going on, but in the end, the product failed. We ran into too many issues with that, and had to call it off. Instead, we started from scratch for this another attempt at the same functionality and this time, it worked. This was a big win for the company and it was celebrated with high levels of marketing, advertising, etc.
We’ve so many other beta features - Prerendering, Split Testing, etc. There’s a reason why we’re calling a feature, beta - because it’s not tested for production work loads, there are chances to fail. If that’s not understood by this point, I’m sorry, but I’m not sure what more to say. You can call that an excuse, but by ignoring the warning of a beta feature (and Eleventy itself saying that their Netlify Edge setup is also experimental), you’re choosing to dive into a risk factor.
You’ve also been complaining about we charging for that feature. Can you please let us know, how much we’ve charged you in your past 6 months for Edge Functions? I’m willing to bet, that’s $0! Please stop spreading incorrect information. On the Pro plan that you’re, you’re getting 15m invocations for free, are you actually saying you ran out of those and got charged?
Calm down? What part of my post was inflammatory? We have a site that can’t be launched, and no indication as to why that is the case, or how to work around the issue. Anyone would be stressed in this situation.
Regarding my comment about charging for the feature, this wasn’t about dollars spent. You are correct, since we haven’t launched yet, the feature has cost us nothing. But seeing pricing information on the feature gave me a false sense of comfort, thinking it was overall ready to use outside of a few kinks.
At this point I would happily pay $ 1,000 / mo to get this launched. Would upgrading to Enterprise solve this issue? I read that your Enterprise customers run on a separate edge network, would edge functions be more reliable there?
Calm down, not in the sense of you’re inflammatory, but in the sense, you’re making too many posts about the same issue.
If we knew the cause of the issue, we have no reason to not share a workaround. But, that’s not the cause at the moment.
Hard to say as we don’t know the cause of the issue. I believe, not but only the devs can confirm.
Also, not sure if this helps you but I’ve pinged the developers to skip the wait time and get to your case looked at as soon as possible.
Thank you so much. Since 11ty Edge was the root of the issue last time we experienced crashes, and since it’s the one doing the most work as far as CPU and memory footprint, I think as a first step we’re going to remove the 11ty Edge function altogether and replace those blocks with client-side logic, but keep all other edge functions handling access control and routing of requests based on user state. Those are very small and fast, hopefully that’ll help.
That’s a good idea, I suppose. Let us know how that goes.
Removing 11ty Edge function seemed promising, we did not experience any of those errors for over 24 hours afterwards, but today it started happening again when browsing in Arizona. Can’t reproduce here in Panama but I received these screenshots from team members in AZ.
Thank you for those. These errors are not unique to your site. The devs have already engaged into discussions with the upstream provider on how to make this better. There is a lot of work going on into improving the logging experience at the moment, so hopefully, next time this happens, you can see the error yourself (or at least we can see some useful error message in our logs) - none of which is happening at the moment.
I just wanted to add I’ve also reported this sporadic problem to Netlify as of recently (the past couple of weeks or so.)
I think my test NextJS website using general middleware (edge functions) also cause these timeout problems.
It’s unfortunate there’s no visibility into the logs for why these Edge Functions fail, but I’m open to suggestions on how to gain that visibility.
Hey there, @Joroze 
Thanks so much for following up. I want to assure you we are actively investigating this and will follow up on this thread when we have more information.
Hi all,
We have identified a problem residing in an upstream service that is causing these timeouts. Presently we are working with the provider on this issue and are waiting for a code fix from them. We apologize for the inconvenience and are doing as much as we can to aid the provider in the restoration of service.
