Sporadic errors: "This edge function has crashed - The deployment failed while serving the request"

Support
31

I’m also running into this regularly, but inconsistently - without anything in the log when I click through. The latest ID was 01GAPWE3EATMNH84T36JSGQ571, and a re-deploy of the site seemed to fix the problem for now. But this is not the first time a re-deploy was needed to fix it.

I’m not sure if I should be removing edge functions from my site until there’s better reporting? Or if this should be resolved soon? I did notice that my previous two builds were both triggered by webhooks, if it’s possible that’s part of the problem? I’m not sure how to debug further, when the issues are so inconsistent, and there aren’t any logs.

When it works, it’s a great feature!

(My edge function is generated by Eleventy)

33

Hey @mia,

When you say a redeploy fixes the issue - does it fix for the deploy permalink for which it was failing? That’s what seems to happen at the moment, but wanted to confirm. I visited the deploy that was active during the request ID you’ve sent, and that deploy seems to be working. This should not have happened if the deploy in particular was a problem. But since you say redeploying fixes it, I don’t see why redeploying would fix an issue with a deploy that’s already done, unless it’s a system-issue and re-deploying it fixes it “coincidentally”.

34

It seemed to be fixing things when I posted - but then I found I was still having regular and inconsistent failures, even on the new deploys. I ended up removing the edge function for now, because I couldn’t figure out any better way to debug the issue.

35

We’re experiencing the same issue, getting edge related errors sporadically. We had a local reference to 11ty edge that broke out of the blue, replaced it with a CDN reference, we thought we were out of the woods, but looking at our monitoring logs we’re still getting random errors. Can’t be reproduced either, the same request with identical headers runs fine for a while, then crashes again.

36

In our situation, it is possible that those errors are related to cold starts. After deploying today I kept getting that generic edge function error, still no log, but after sending a bunch of requests, about 5 minutes after deployment, pages starting loading.

It’s a problem though, since we’re getting ready to launch. We’re using 11ty edge functions btw, although I don’t know that it’s related. Our site used to work great, then there was a breaking change that seemed to be related to the 11ty edge dependency, we moved that dependency to a CDN URL, the site started working again but we’re encountering these weird random errors.

The key problem I think is lack of visibility. Errors and no log.

Any idea when Edge Functions are supposed to be out of beta? We’re new to Netlify , is it pretty common for your guys to publicly advertise and charge for a new feature yet keep in beta for months?

40

Hey there, @slegay :wave:

I just wanted to reach out and close the communication loop-- I see that your questions have been cleared up in the helpdesk with our Director of Support. Thanks for following up there.

If you have any further questions let us know. Thanks for choosing Netlify

  • Hillary
41

We are still getting the error occasionally. Definitely happening after a fresh deployment, for a few minutes, then all pages load fine.

42

Over the past 3 days we’ve been seeing very erratic behavior. Just clicking around the site, the same page will render fine in sub-200ms response time a few times and then crash with a generic edge function error and no log. A few minutes later it’ll work again. Seems to be happening on multiple edge locations - we’ve experienced this on our tests in west USA and Central America.

Also, my questions have not be cleared up in helpdesk. I did get a reply, but the answer I got was ‘We don’t know’. It appears that tech support does not see anything in their logs either.

43

We don’t have any additional, or hidden logs for that at the moment. So, what you’re seeing, is the same error we’re seeing in our side of logs. So, this is partially true.

Edge Functions will improve over time - every product needs time to mature. When that happens, hopefully we’d have a different answer.

Consider Netlify Functions for example. It’s a mature product at this point. I don’t recollect we having to say, we don’t know what’s happening with that one for a long time now. Edge Functions will catch-up soon, but it needs its time.

44

This wasn’t an attack on you, I believe your team is doing what it can to support the product. But Hillary’s post made it sound like my issues were solved in a direct support thread. It was not. The facts are:

  • We have an unstable site in production that shows pages about 50% of the time. The other 50% of requests (from the same client, in the same edge region) crash with a generic edge function error, which makes the site so unreliable it is unusable.
  • The problem is not reproducible at all on Netlify CLI.
  • There is nothing in the log that your team or I can see.
  • I am not the only customer reporting this issue.
  • The advice I received from the director of support was to stop using edge functions, and you offered a refund on my pro account. But there’s no known workaround for us to be able to launch our site as it is.
  • In order to launch I need to convert all of our authentication, access control and personalization from edge functions to client-side JS and serverless functions.

This makes the Edge function feature unusable at this time, with no clear timeline as to when it will actually work. I understand that it is our fault for taking a gamble on a new technology 6 months ago, hoping that by the end of year the feature would be robust enough to launch our site. Typically, products released as public beta tend to be pretty functional, particularly with so much marketing around the original announcement back in April. Nothing like this has ever happened to us when using beta products from AWS for instance. And the goal of a beta release is usually to gather feedback from early adopters and put them in fairly close contact with engineering to resolve issues. Your early adopters will be the ones advocating the feature in the long run, we take risks by investing time testing and building on top of your new product, and expect to be treated as partners. This hasn’t been the case, there has been many reports here of crashes and errors and no word from engineering as far as I can tell.

I made the wrong call, it was a costly mistake on our end, but it’s time to move on and put this edge stuff behind us. It’s unfortunate, this is a very promising technology.

45

I agree and hear you! You can be rest assured that all of this feedback is not going waste - it will help us shape out future product launches.

As I’ve mentioned in the other thread, for our initial attempt at launching Edge Handlers (that was the name back then), we were having private beta testing in which the testers were able to directly contact the engineers. If I recall correctly, even for this one, we had testers, but they were mainly framework devs like Nuxt, Svelte, etc.

You’ve taken a part in public beta. Providing over 3 million Netlify users a direct communication channel with the devs would not be scalable and possible. Yes, not everyone will reach out or have issues, but even if 1% reach out, that’s still a huge number! However, every thread that you make, is being monitored from what I can see.

Like I’ve mentioned in your other thread, your feedback on our Docs page was also noted and discussed by our Product Manager. So, you’re not beating yourself against a wall - your complaints are reaching to the devs.

I really hope I had anything else to share with you, but I sadly do not. But I hope you’d be able to reconsider Edge Functions when they’re out of beta.

1 Like
46

I am experiencing the exact same problems on my site. And, I am only running the default edge functions (netlify-handler, netlify-obd-handler, and _ipx because the site is built with Nextjs). Would just like to add a “+1” to this thread.

47

Like aleksati this has been happening to me too and I don’t really use edge functions for anything except the default ones because it’s a NextJS app

49

On a higher level, we have identified a problem residing in an upstream service that is causing these errors. Presently we are working with the provider on this issue and are waiting for a code fix from them. We apologize for the inconvenience and are doing as much as we can to aid the provider in the restoration of service.

But since we’re generating the Edge Functions for Next.js sites ourselves, we are looking it we can work around this for the time being.

50

Is there any update on this? I’m experiencing the same as described by many above, in that it seems to work in some places and not others.
We are using an edge function to determine location and store accordingly. This is working when I view the site, however, Netlify’s own function to screenshot the homepage of the site for the admin, as well as other 3rd party tools, like Facebook debugger, are reporting 500 errors and displaying the Netlify error:

image

image
image1319×1119 87 KB

The SEO and sharability piece is of great concern

52

In general our edge functions are not experiencing many consistent stability issues at the moment. Are you sure you’re catching all possible exceptions in your code?

53

No - I’m not sure.

I do have try/catch method wrapping the entire logic and if failing, just allowing it through in hopes that would eliminate the error, however, this has had zero effect. I would have thought that a script wide catch would have been that “catch all”.

54

Can you paste your edge function code here? Do you see anything in the edge function log?

Maybe its a coincidence, but we’ve had better luck with edge functions since removing all library references and only using Web APIs provided by Deno, limiting edge functions to validating cookies for access control, routing request, and a couple regex response transforms.

We also do not make ANY network calls in edge functions. Anything requiring access to external resources like a Fauna DB query or fetch requests resides in serverless functions.

56

@slegay - edge log is blank. The function is merely geo locating the user and storing to cookie.
Heres the code:

export default async (req, { next, cookies, geo, log }) => {
    const res = await next();
    try {
        const headers = req.headers;
        const host = headers.get('host');
        const contentType = headers.get('sec-fetch-dest');
        const currentLocale = geo.country.code.toLowerCase();
        const path = req.url.split(host);

        const locales = [
            {countries: ['at','be','bg','hr','cy','cz','dk','ee','fi','fr','gr','hu','ie','lv','lt','lu','mt','nl','pl','pt','ro','sk','si','es','se'], path: '/en-eu/'},
            {countries: ['gb'], path: '/en-gb/'},
            {countries: ['it'], path: '/it-it/'},
            {countries: ['de'], path: '/de-de/'},
            {countries: ['us'], path: '/en-us/'},
            {countries: ['ca'], path: '/en-ca/'},
            {countries: ['au','nz'], path: '/en-au/'},
        ];

        if (contentType === "document") {
            const siteForLocale = locales.find(locale => locale.countries.indexOf(currentLocale) > -1);
            // log(geo);
            if (siteForLocale && !path[1].startsWith(siteForLocale.path)) {
                // log('Detected better location');
                const suggestData = [
                    geo.country.name,
                    siteForLocale.path.replaceAll('/', '')
                ];
                const cookieString = suggestData.join('|');
                // log(cookieString);
                cookies.set({
                    name: 'fc_suggest_location', 
                    value: cookieString,
                });
            } else {
                // log('Clearing cookie');
                cookies.delete('fc_suggest_location');
            }
        }
    } catch (e) {
        // cookies.delete('fc_suggest_location');
    }

    return res;
};
57

Thanks so much for sharing the code! Our dev team should look at this for us tomorrow and will hopefully have some feedback.

1 Like