Questions about Netlify Identity Serverless functions

I’m trying to understand how to write a serverless function that will be run once for new users. According to the docs, we’ve got: validate, signup, and login.

I do not see a distinction between validate and signup. The description for both are literally the same, “Triggered when an Identity user tries to sign up via Identity.”, except that signup doesn’t run if social auth is used. Why would I use signup ever? I may launch w/o social login, but if I add it later then my signup serverless function won’t be used for those signups, so the only safe option is validate. But if both exist, they must serve some different kind of purpose, right?

Also, I see that for a new user, login isn’t called, even if the user gets logged in (because they sign up and validate), so given that if I need a hook to “new user on the system”, I need to just use validate, right?

And I know it’s been said… but please, please, please add documentation that shows the ‘shape’ of the data sent to these functions. From what I can see, I can’t test the identity serverless functions locally, so I have to test on the live site with tabs open to my app settings and manually watch the logs. This post, Identity-signup Event Triggered Functions, from over a year ago, is helpful, but should be documented in the main docs, right?

3 Likes

As a followup, it looks like validate is called before the user confirms. Do we allow users to login who haven’t confirmed yet?

Basically I need to know when a new user is created and valid.

1 Like

And shoot. So I just tested w/ social login. It worked beautifully in terms of the front end, but none of the serverless functions were called. That’s a bug, right?

2 Likes

Ok, not sure if it makes sense for me to keep commenting, but I keep noticing things. So this morning I see that validate’s description is different, “Triggered when an Identity user tries to sign up via Identity”. Whereas signup is “Triggered when an Identity user signs up via Netlify Identity.”. I still don’t quite get the distinction. So is validate called before signup and if so, what’s the point of both of them? What’s the use case of using both in production?

Also - if social login doesn’t call signup because, in theory, they already signed up for their social network, I think this is incorrect logic because they are still becoming new users for the Netlify site. Again, I’m just guessing here as I haven’t gotten a reply yet as to why none of the events run on a social login.

1 Like

Hey @cfjedimaster :wave:t2:

Thanks for all the context and I hope you’re enjoying playing around with Identity and Functions and how they work together :slight_smile: I’ve personally done a lot in this space recently as well. Let me try to get through some of your thoughts here.

Identity Event-triggered Functions - Of these three hooks, none are called when the user signs up via 3rd party services (e.g. login from Google, GitHub, etc.). While it’s not necessarily written out in the docs, there are other threads here on the Community that get a bit deeper into that, but the idea is that you’d probably just make a normal Function and kick it off from a web hook provided by the 3rd party. The workflow for these Identity-Functions is somewhat geared at being able to validate users before and during the signup process as to potentially block them from creating an account, but if you’re using 3rd party auth, by doing so you’re inherently unblocking folks from making accounts (3rd party registration is open to anybody with an account on the third party service)(plus, the way 3rd party auth works, you have no user metadata to filter/gate your account registration on anyway…). Presumably, all of the processing you’d want to do when using a 3rd party auth service would come after the user gets an account; which is best kicked off from a web hook on the 3rd party side because the 3rd party “sign in with” process may have its own steps to work through before hitting ‘done’. 3rd party auth services should allow you to hit a web-hook after the user is done signing up; just point that web hook at your own Netlify Function for your site and you’re free to do whatever sort of post-sign-up process you’d like :grin:

Validate vs. Signup I’ll agree with you here - the docs aren’t super descriptive on what’s what. Here’s my understanding - Validate happens once the user fills out the registration form, but before the confirmation email is sent. If you return a 200 or 204 from identity-validate, it will go ahead with the email and the user can confirm their registration with the unique token contained in that email. The impetus here is being able to filter out users by their signup handle / other signup metadata you may have in your custom signup form. E.g. you only want to allow users from @mydomain.com email addresses to be able to sign up for an account on your site. The identity-validate function should let you control that really easily.

As an exercise in fun and brevity, that entire Function could be as simple as (lol this is ad-hoc written and untested, FYI):

// identity-validate.js
exports.handler = async (event) => ({ statusCode: /@example\.com/.text(JSON.parse(event.body).user.email) ? 200 : 403 })

As an aside, I think you have full access to the registration content that was submitted as part of the identity-validate payload, so you could hydrate user metadata here too. E.g. if your custom user signup form includes a phone number, since that would be a meta-field in Netlify Identity, you could add it here. You could also add it in identity-signup (below) so both work fine, but just calling it out since sign-ups often require more than just email/pass

On the other side of the coin, identity-signup runs after the user has completed their signup and successfully processed the valid confirmation token from their email confirmation. I like to consider this as the best place to handle “setting up user default roles and metadata” – and have used this myself a number of ways, but most commonly to assign users with a ‘default role’ for role-gated content. As mentioned above, you could also populate other metadata here from the registration form that was submitted, but I’d have to test that out myself to confirm the shape/workings of that. Here’s another example in brevity for assigning a default role to all new users:

// identity-signup.js
exports.handler = async (event) => ({ statusCode: 200, body: JSON.stringify({ "app_metadata": { roles: ["member"] } }) })

Data-shape Yeah :confused: I’ll agree with you, my friend. The docs could be better here. If they were open source I’d contribute myself :rofl: but I also agree that @futuregerald’s comment here (same link as yours) was helpful… but understanding the full data shape is tough, and I agree that you have to sleuth through some demos using the prod identity UI and Functions logging to get where we need to go. TBD on that for now; although it’s in my list of things I’d like to write about.

I hope all of that helps :slight_smile: Please let me know if you have more questions. More than happy to help on this topic. I do find that there aren’t nearly as many posts / info on the ‘deeper dev’ of the Netlify platform (e.g. Identity and Functions and Large Media – some of the deeper stuff) so I’m always happy to increase the volume of written works on these things


Jon

1 Like

Holy smokes, thank you for the incredibly detailed response!

Social Login: I get your logic but… it still feels wrong to me. Consider this. Yes, I have a Google account, but if I visit RandonNetlifySite, I do NOT exist as a user. If I go to the app settings and look at Identity, I’m not listed there. I have to click the login/signup button and go through a process before I’m “known” to the site. Something on Netlify’s site does this. It says, “foo@foo.com is a new address, add him or her to the list of users”. If Netlify can recognize a new social login, doesn’t it imply that I should be able to access that and do something with it?

Most likely I’m going to have to live with it, and that’s ok. I disagree, but at least now I know the logic. But please, please, Netlify add information to make this clear!

For me, I just need to know when a new user is here and record them to Mongo. I’m going to write a new serverless function that is run after login on the client side. It will be “slightly wasteful” since it doesn’t really need to be called for every login, but I’m totally fine with that.

Data Shape: I just discovered “netlify functions:invoke” and I’m playing with that now.

1 Like

Quite welcome! Often I write extensively in hopes it’ll be found by many more in the future - just as you and I found that older thread :slight_smile:

Social Login - yeah, I totally get it. I should asterisk too - I can’t personally confirm or deny the behavior, especially with identity-validate and even moreso, identity-login. I haven’t played with the 3rd party auth and technically the docs only say that identity-signup is skipped when using 3rd party; so it’s entirely possible that validate and login do work. If you’re using login with 3rd party, you may just have to build in some logic for “if these fields aren’t already set for this user, set them” since signup was skipped. Food for thought :bread:

Either way the docs could certainly be better and code samples would be helpful 100%.

Data Shape - ntl functions:invoke is helpful for kicking off the function, but do be aware that you need to provide that hook with the payload to pass to the function, and that in it of itself can be tricky :stuck_out_tongue: to meet parity with the events on Netlify Prod you may or may not need to add another wrapping layer. I should write about this :laughing:

Also, as a quick tip / trick (which I have yet to write about but will give to you anyway because it’s mega helpful) - if instead of running netlify dev, you run npx --node-arg=--inspect ntl dev, it’ll expose the Node debug port for the Functions server running locally. You can connect to that port in VS Code natively and get full break-point debugging and context for your local functions. Super useful for ‘normal’ functions. Still useful for event-functions when you use invoke in another process to throw data at the function, but again, be careful with the shape of that data to make sure it matches prod.


Jon

1 Like

Actually, the invoke thing does send mock data, but only for Identity functions. I checked the source.

“Web hooks” - I literally just now discovered that Netlify supports webhooks for identity stuff. I think you mentioned web hooks before, but I thought it was on the Google side. This is not documented anywhere I could see in the Identity section.

2 Likes

Oh word! I’ll have to go back and update myself on the source there. I’m a few releases behind, that’s awesome if it sends mock data now :heart_eyes::heart_eyes::heart_eyes:

I was implying Google-side hooks sending data to Netlify Functions (that you write) but yes, Netlify supports out-bound hooks as well. Is that helpful? Would love to understand what you’re thinking about workflow there

1 Like

My workflow is simple - the first time a user logs into my site, I need to record them into Mongo. Why? Imagine later they post a comment. I want the comment to point to the user so I can display their name. Obviously I could store the name as a string, but if they change their name then it wouldn’t update. Also, by having a user’s table, I can get all the comments for a user, all the foo for them, etc. Afaik, Netlify doesn’t give you code access to all your users?

1 Like

As for the mock data thing, unfortunately, its only for Identity. Nothing for deploy events or form submissions.

1 Like

As an update, if you specify a webhook for validate, it works for social login (and I assume ‘regular’ as well). I honestly don’t understand how the serverless function for the event doesn’t fire but the webhook does. It’s inconsistent for sure.

1 Like

This is turning into a great documentation thread :grin: I ended up spending a significant amount of time digging into the Identity / Functions interplay here. Most of this stuff was written by OG Netlify in 2018/early2019 as far as I can tell. Lots of code digging :grin:


A few things to get through below but first let me ask a different question. Does having outbound web-hooks in the UI work better for you than having the same event-driven Function running? Presumably adding a web-hook in the UI just adds a function behind-the-scenes that listens for the same event and just POSTs the data straight out to whatever address you define in the UI. I’m wanting to understand if the outbound web-hooks UI presents a different / better workflow for you than making your own Event-based Functions :thinking:

EDIT: I wrote the above before your response came in (while I was writing the rest) – It would be very strange to me that the UI-based outbound web hooks have different event triggers than the event-triggered Functions. Very strange. I assume the UI-based hooks are just Functions under the hood… so I may check this out on a demo app. I’ve had a great experience with Identity and Functions overall but haven’t played with them much on 3rd party auth.


I want to approach the following as an overview of the Admin/User contexts with Netlify’s Identity (GoTrue) service. It’s critical to understand when designing your auth flows and patterns but really tough to find examples, specs, and docs on.

Actually yes, there is a way to get a list / index of your users! It’s actually built in but to get there we need to understand the difference between Admin context and User context. This is outlined as part of the readme for the gotrue-js package and briefly mentioned in the Functions & Identity Docs and even talked about a bit and implied in this very OG article from Biilman back when all this stuff was releasing. It’s worth taking time to read those, but I might recommend getting through all of this first.

Essentially, any time you’re interacting with Netlify’s Identity service, you’re considered in one of two contexts: an Admin context and a Client/User context. This is fully separate from the concept of user roles, so those don’t matter here. The Admin context resembles the process of logging into Netlify UI and having ultimate control over your site’s user. You’re not logged in as any of them, you’re interacting with the user database as a fully external, administrative person. The Admin context is defined and powered by Netlify Functions. For any site that has Identity enabled, Functions on that site will have a few fields added to the context object (a param of the Function) and one of those fields is an Admin token. It’s short lived authorization token that allows you to execute Admin level commands from your Function - like getting a list of all users. Running a Function (whether by hitting it manually or having it trigger via events [Identity or otherwise]) is the only way to get an Admin token for your GoTrue instance. Since we (the devs of the site) write the code for Functions, it’s considered safe code and granted the same rights to manage the user database as if we logged into the Netlify UI.

  • Getting user info (including user and app metadata)
  • Updating user info (includin…^^^)
  • Kicking off a user invite
  • Creating a new user (kicks off email confirmation)
  • Deleting a user
  • Getting a list of users (haven’t personally done this from the API yet; I think it returns the full object template which includes app_metadata and user_metadata for each user – no need to /GET each one after the index call)

are all available actions in the Admin context.

Conversely, when you’re interacting with Identity from a browser / or other Javascript space outside of a Netlify Function, you’re just a User and you are operating within the context of a logged-in user. The endpoints you can access are limited, you have to log in first, and you can only alter user_metadata (talked about further below).

Now, I’ll say this. When reading through the readme for the gotrue-js package under the “Admin Commands” section, it’s a little confusing that there’s a client-library call shown, then an example lambda method shown right below it. The client library wouldn’t be able to make that call, right? I just explained how only Functions get the auth token to make Admin calls! :laughing: But the admin commands are built into the client package; they’ll just fail unless you manually pass an Admin token from the Function back to the client specifically to do Admin-context actions on the client side. Generally you wouldn’t want to do that, but use cases do exist. Netlify’s own Dashboard UI is one of them :wink: Here’s what I mean:

For a pragmatic example, understand that when you or I use the Netlify Dashboard to manage the Identity on one of our sites, we’re actually using that very same GoTrue-js client aimed at the GoTrue instance sitting behind the site, and Netlify’s core APIs passed an Admin token to our client so that we could indeed manage our Site’s GoTrue instance from our client.

In general, don’t necessarily follow these examples to a tee. They illustrate a situation where you actually have two auth layers - exactly how Netlify works. You and I log into the Netlify Dashboard and get to act as Admins for the lower-layer GoTrue instances (sites). This model doesn’t work if you’re talking about a single-layer (one site; one GoTrue instance) without some strong guards. You generally don’t want to expose Admin tokens to clients that are users of the same instance. It can be dangerous.

Instead, use these as a loose guide for implementing Admin level functionality via Functions - not passing the Admin token back to the client :+1:t2: you can go a long, long way leveraging that well.

Phew, that’s a lot. :sleepy:


Some other things that should be called out when using Functions and Identity -

user_metadata can be updated directly by the logged in user from the client side, but app_metadata may only be updated from Admin context. Now, a key note here is that both buckets are visible on the client-side, so don’t store highly sensitive or non-user-specific data in app_metadata thinking it’s safe; it’s visible. It’s just read-only from the client. You may only update and/or alter that data from Admin context (e.g. within a Function). That can be used cleverly for sure :nerd_face: This is proved out and documented in one of Netlify’s demo repositories identity-update-user-data which is actually a fork of @futuregerald’s and also well described on the gotrue-js readme here:

Users can update their user_metadata field. To do this, pass an object to the attributes.data key with the fields you want to update. Updates to a users app_metadata must be performed from a secure environment, such as a Lambda function. For examples on updating user and app metadata, see netlify/identity-update-user-data.

Outside of that, there’s a react package (not netlify-identity-widget) that makes life easier too

This article “How to Build Authenticated Serverless JAMstack Apps with Gatsby and Netlify” was written by @swyx - who, though I don’t know all the fine-grain specifics, built a large portion of the Netlify Functions, Identity, and Dev stack (though is no longer with Netlify :frowning:). While it’s geared at Gatsby, the article gives insight into some of the cooler capabilities of Netlify Identity, including the last section “Authenticated Netlify Functions” which is great. Essentially, you can use an authedFetch out of the box to automatically inject your user token into your request so that the Function you hit (which has Admin context) will have a properly hydrated user object to see who called it. That can be helpful no doubt. Do note - while both the Netlify Identity Widget package and @swyx’s own react-netlify-identity packages are build on the gotrue-js library, they are separate wrappers.



Haha yes, I see that now. Hilarious. Mock data only for identity and, helpfully, authenticated requests to local functions! Nonetheless still helpful. Netlify Dev is still technically in a beta state I believe, so it’s incredible all the things it can do. Just something to keep in mind :nerd_face:


I hope that helps and makes sense. I know there’s a lot here but hopefully a solid read or two and things start to make more sense. :100:

3 Likes

Tagging @Dennis here to request a read-through and confirmation on the content of this thread :pray:t2: and @perry for visibility.

Earnestly, I think this thread may be the best collection of links to docs, info, and general how-to for Identity and Functions? Not sure what to do with that information but hopefully it can be made more visible for folks in the future?


Jon

1 Like

Well to be clear, these are the “if you name a function a certain way, we will call it” functions. And I tested multiple times, the identity-validate serverless function was not called on a social login. If I define a webhook for the same thing, and in my test I just pointed to a random function, it does fire on my first hit with the social login.

So that’s cool - but is there no way to get one user? So imagine I’m fetching a set of records from Mongo and each record has a useridfk. If I can’t get one user, or N users, I have to get all users and loop to associate them. If my site has thousands of users (I wish :wink: then this would be pretty wasteful. It is good we have API access to it - just curious there’s no way to get one or a subset?

1 Like

So interesting. Great findings. :100:

Whoops - that was a typo. Fixed, but “like getting a list of all users”

Yeah you can totally get a single user.

Probably should’ve said “Getting [a single user’s] info” there - but the last bullet is actually the only multiple-user action. All of the rest are single user.

The list request also supports filtering too. Under the hood, it’s a %{arg]% style string matching on either the email OR full_name fields but this isn’t implemented in the GoTrue-js client package - you would need to do that manually from a Function (which you probably would anyway) as shown here but adding a query parameter (not shown in the readme I just linked, but the GoTrue code tells all :wink:) like:

// Get all users with the same email domain as the user that called this function;
// return count to user 
import fetch from 'node-fetch';

exports.handler = async (event, context) => {
  const { identity, user } = context.clientContext;
  const adminAuthHeader = 'Bearer ' + identity.token;

  // May need to encodeURIComponent() on this
  const filterArg = user.email.split('@')[1]
  const usersUrl = `${identity.url}/admin/users?filter=${filterArg}`;

  // Do better error handling if doing this for real 😉
  const data = await fetch(usersUrl, {
    method: 'GET',
    headers: { Authorization: adminAuthHeader },
  }).then(res => res.json())

  return {
    statusCode: 200,
    body: JSON.stringify({
      count: data.users.length
    })
  }
};

:slight_smile:

1 Like

Interesting. So… there’s still a part of me that feels like I should have a copy of my users. So imagine the hypothetical site of films with comments. Also imagine films have tickets so there’s orders. My Mongo collections would be: films, comments, orders, and users. comments and orders would FK back to users.

But… I’m tempted now to let Netlify just be the one source of truth for users. FOr things like getting orders, it will just be for one users. It’s only going to be a ‘problem’ when I do things like, Get Comments for a Film, because then I need to do the lookup on the FKs to show names.

1 Like

Yep totally, but I think there’s a few ways to handle that.

If you have a show page for a film, maybe on the bottom you want to load the last 5 comments on that page. Just have the front-end load those comments once the page has rendered (e.g. load async) and have a Function for loading those. The function might reach out to your comments store, grab the last 5, then hit the Identity endpoint for each of those 5 users to get the full name, etc. – basically using a Function as your JOIN point between the two data sets.

Alternatively you could consider the Netlify Identity system purely for auth and have all of the comment data stored on your DB. The comment content, the name of the commenter, etc. (everything to generate the UI for the comments), then only use Netlify Identity as a means for authorizing whether someone can create a comment or delete theirs. Have a hook for if the user changes their name - that can go through and update your DB for all the comments with their name, but treat your DB as a flat data store itself and don’t consider the Netlify Identity system as a FK to your DB but rather a separate system auth-guarding changes to your DB. Keep just enough information in your DB to tie the two systems together in the rare cases you need to (name changes), but otherwise keep them disparate.

It’s a bit of a tough idea to grok (at least for me) but I think it’s probably more performant :+1:t2:


There’s also the total other route of actually storing comments statically and having them kick off builds of your site. A super cool avenue. More food for thought :slight_smile: JAMstack Comments | CSS-Tricks - CSS-Tricks

1 Like

Thanks. This is for a series of posts I’m doing on converting an older Express app to Jamstack. I’m going to be writing up the latest later today or tomorrow and will be pointing to this thread.

1 Like

You bet :+1:t2: and glad to hear it. Will for sure give it a read :slight_smile: Post back!


Jon

1 Like