Pardon the pun - if you have ideas around how we can make #netlify-platform:functions even better (feature requests only, please, no current support issues here, make a separate topic for each) please let us know and we’ll look in to it
Are there any plans to introduce scheduling lambdas? I’m using Zapier to trigger them at the moment which is fine but would rather have this functionality within Netlify instead of another service.
They could live inside the toml file?
serverless example:
{
schedule: {
rate: 'cron(0 23 ? * SUN *)',
},
}
[build]
functions = "./functions"
[functions.${nameOfLambda}]
schedule: 'cron(0 23 ? * SUN *)',
hi @omonk - i don’t know of any plans for this yet, but that doesn’t mean it is a good idea. I’ve moved your post here here so we can look into it and consider this
Any traction with this? I too would like to consolidate to Netlify but this is holding that back a bit.
In order to schedule a function to run we could use a cloud watch scheduled event, which calls a lambda, which calls the Netlify function. Defeats the purpose to have the Netlify function since we have an AWS lambda to manage separate anyway.
Just a simple configuration allowing us to specify a cloud watch trigger schedule like is done in a serverless.yml would be fine and I suspect not a huge lift since your already deploying the lambda for the function as it is.
I don’t know of any specific plans to provide this service. But this also is a pretty surprising feature request. Our functions are designed to run at browse time, synchronously, with a web request. Your use case sounds different - maybe you could elaborate on it? Other uses than the one we intended are of course possible, but that feature was designed with a very specific use case in mind and that use case (visitor browsing your site and needing some dynamic information right now) would not benefit from scheduled execution
We usually get that feature request for builds, where scheduling a build to publish a blog post or regularly pull in new content from the CMS is more in line with what we were aiming to enable, but this is the first I’ve heard of anyone wanting to schedule a function so I’d love to hear more about your use case
A cron server or Zapier’s scheduled zaps are our two suggestions for implementing yourself, which I realize does not actually solve the problem in the way you’d prefer.
I can understand the use-case it was built for but would argue that there is a need for time-based triggers of Netlify Functions as it’s in my opinion the only missing piece of a complete JAMstack architecture.
Assumption
Assume my goal is to eliminate completely any backend server in exchange for a statically hosted front-end, serverless functions providing all traditional server-side abilities, and persistence provided as a hosted service for me, then where do scheduled jobs fit in this architecture?
Consider
As more and more is possible on the client side (rich SPAs), and more and more backend features are made serverless (FaaS, PaaS, RDS, Graph.cool | Prisma | Hasura hosted), the traditional backend server goes away. One of these backend servers is where we would traditionally run time based tasks scheduled with cron.
Example Use-cases
Scheduled jobs are a necessary part of many application architectures but let me mention a few that come to mind from our requirements:
- periodically tracking active shipments with UPS since UPS does not offer an event-based solution
- alerting when an expected state change has not happened after a set amount of time (the lack of an event + duration = new event)
- action that must happen on a specified amount of time prior to a future state change which will happen on a schedule (preparing a shipment for an order fulfillment in the future)
These small bits of logic fit into a lambda based micro-service architecture well but miss a clean and easy way to trigger them on a schedule without leaving the Netlify platform.
Solutions
- Zapier (pricey if all you’re doing is scheduling an HTTP invoke of a Netlify Function)
- IFTTT (free)
- CloudWatch (free)
- EasyCron (pricey for just job scheduling)
These all provide viable solutions, you are correct in mentioning them, but they don’t easily fit into the CI/CD flow on Netlify. I’m not arguing that if Netlify doesn’t provide this functionality, the community is left without because the above solutions work fine albeit with trade-offs regarding deployment. Rather I’m simply suggesting that the added complexity of leaving, the otherwise complete, Netlify stack for something as trivial as setting up a scheduled invocation of one of the Netlify Functions could be removed with a configuration option on Netlify itself. A convenient way to implement this would be a new section in the toml file as @omonk suggested. On the Netlify side this toml entry could simply be translated into a CloudWatch scheduled event triggering the Lambda that backs the Netlify function specified.
Thanks for taking the time to read all this!
Hi @rob,
Thank you sooo much for providing your perspective. I agree that this would be an awesome feature to have and a problem that I now solve using third party services. It would be cool to do it within Netlify.
+1 to everything @omonk and @rob have said.
I watched two Gatsby/Netlify related livestreams recently which discussed cron scheduling of posts and saw another related to refreshing of external API data. In both of the Gatsby streams I mentioned and pointed to Netlify’s own www-post-scheduler, however it hasn’t been touched since 2017—uses serverless and AWS—so Zapier was used instead.
Perhaps that project is a good start for a Netlify functions/dev cron refresh?
Peace
After reading “Modern Web Development on the JAMStack,” it occurs to me that writing the same content to multiple static pages in a site is only one step better than writing the same content to multiple dynamic pages in a site. It would foul up local previewing, but wouldn’t a more efficient approach be to have snippet injection based on other HTML tags than <head>
and <body>
? For example, <header>
, <footer>
, <nav>
, <article>
, and <section>
could each be filled from the CDN with snippets, couldn’t they? It seems this could be tremendously useful while we’re waiting for whoever does the HTML spec to pull their heads out of the ground and make it possible to include HTML files as easily as we currently include image (and other) files. We shouldn’t have to resort to PHP includes (or JavaScript!) to accomplish this simple task.
Heya @gregraven! Hope you don’t mind but I edited your post slightly since the unescaped HTML made your post look like “For example, , , , , and could…” so I was a bit confused until I dug in to your source code.
I don’t think we could really support any of those tags, since they can all (AFAICT, though apologies if I missed something) legally appear multiple times in a document. Our current implementation is pretty straightforward and tied to single-use-only html tags. Or maybe you meant more “some dynamic content” - in which case I’d run a function one time on a page, and then use its results do the “injection” myself clientside wherever I wanted while painting the page.
Could we support other elements with our current implementation? Of course! Could we do create a feature to do the second thing I mention more automatedly? Sure! I’ll be happy to get a feature request filed for us (which means the product team will see it fand consider adding it to the roadmap, and we’d notify this thread in followup if that happened). But, I’d need you to first flesh in a bit of how you’d see it working, for instance: how to handle multiple possible interpolations in the same file - just the first tag’s occurrence?
Yes, just the first occurrence. Thanks. I see it working similarly to document.getElementById().innerHTML in JavaScript, but faster.
Thanks, added it to an existing feature request, but as I mentioned, I don’t see it happening since behavior wouldn’t be as intuitive. However I’ll follow up here if things change!
Came to this post, because I’m about to solve this issue myself.
What third-party service do you use for this task?
Hi,
Assuming you were talking about scheduling, we use Zapier for that here. It works pretty well for me.
Would also love to see custom runtimes! It would make integrating/reusing existing code much easier (instead of having to re-write in js/go)
Something that has recently been annoying me about Netlify functions is that the HTTP headers set in the _headers
file do not get returned with functions. I setup a /*
rule with CORS, etc headers but I have to add them to each Lamdba functions response manually which is annoying. I think it should default to the _headers
file and then if the Lamdba responds with a conflicting header it should override.
I’d love some useful stats on functions usage of a project under the new analytics tab.
Stats I’d find useful are:
- number of times each function is called (like most visited pages in current Analytics)
- average time a function runs
- error counts (like the missing resources in current Analytics)
- total function consumption (like bandwidth usage in current Analytics)
Hi,
I would like to see report-uri / report-to functionality for csp and expect-ct policies.
hi tmoberg, can you outline your use case with a little more detail? The more info we have on how this helps you (and others) the better. thanks!
I’m also going to move this post to our feature request thread for functions.
Hi Perry,
I am thinking of reporting as a serverless function, where violations of csp, expect-ct and other, future policies are handled with netlify functions and user-controlled endpoints.
I don’t think I can be more precise.
Thanks,
Tom