Hello, for deploy 651af90011a21500080b9365, I have a function “ScheduledReports” that seems to have been invoked twice at 0:00 AM (execution id dea9dfff) and 0:01 AM (execution id 82f6e319) on October 16th (GMT)
I see in the logs the two executions having different container/execution ID’s, and starting within a minute of each other :
Cron expression says it should only run once every 10 minutes :
Any ideas why this function got re-invoked ?
Potentially good info:
My code is supposed to update our “last_run” variable in our database for the report in question, which would prevent it from being executed twice, but the code to do so only runs at the end of the function after the report is sent successfully. This time, however, the function ran twice, and the report was indeed sent twice, even though the second invocation was 16 seconds after the 1st finished (and thus that variable should have been updated.
Is it possible the function had some sort of failure/error and lambda automatically re-tried ?
The logs show that the 1st execution finished correctly without any errors…but maybe I missed something in my error catching?
Howdy @danielthedifficult ,
Are you using Supabase by chance and, if you are, could you check to see if it cloned the cron job locally causing it to trigger the job twice, once from production and once from your local machine?
If that doesn’t review any quick wins, could you share your site name so that we can take a closer look? Thanks!
Unfortunately no, no Supabase integration. This job is triggered via netlify.toml in cron syntax and has been working great for months.
Site name is gv-dash
Thank you l!
We had an internal error with the first invocation, so we retried invoking it. We try 3 times before giving up and waiting for the next schedule.
Hi @hrishikesh, that makes sense.
Can you tell me more about the internal error so I can take that possibility into account in my code?
I.e., did the container ‘kill’ before the end of execution? Or did something else happen?
The reason I ask is again because I do track the most recent successful completion of this scheduled execution in our DB, and even if you re-invoke, I shouldn’t re-trigger the same report twice… but somehow this update never made it. Can you share any additional details?
In our logs, I see that the request timed out when waiting for response headers. On our CDN, you need to at least send the headers within the first 28 seconds of the request, the rest of the body can continue streaming later. But that didn’t happen here according to the logs. So we termed that as a failure.
Hi @hrishikesh, getting warmer…
Thank you for that information. I think i’m a bit confused because it’s a Scheduled function.
Does that mean that the function is supposed to send headers back to your internal invocation function/feature?
Is that 28 second timeout related to this?
How could/should I ‘acknowledge’ the run by returning response headers before the end of the function?
All scheduled functions are required to return a statusCode of 200, and even that’s enough to return headers.
This is not always a user error and could be very much just a system/network error. I’d say, don’t overthink it or try to engineer a solution for this as long as your functions are working as scheduled. Just always make sure to have a protection in case your function is invoked successfuly twice, so it doesn’t do the job it has already done (like modifying a database value, for example as that could lead to disastorous results depending on the case).