Select Which Environment Variables to Include in Function Build

Support
, ,
1

Overview
I am exceeding the 4kb limit on function env vars. Is it possible to include/exclude env vars by name from function bundle and deployment?

Details
I have a large cert (3.2kb) stored in a Netlify UI env var that I am trying reference in one of my lambda functions. I already have 36 existing Netlify UI env vars (2.1kb). After adding this new large string, I am exceeding the 4kb limit for the lambda env vars. It appears that all env vars without the REACT_APP_* prefix will get bundled into EVERY Netlify Function, even if they aren’t used in the function. Despite only one of my functions requiring this larg cert, all functions are built with it included adn thus deployment of my site fails with this setup.

Suggestion / Question
I only need this env var in one of the Netlify Functions on my site. Is it possible to put a configuration in the function’s directory that could explicitly state which env vars to include in the build? Could I do the opposite of that and specify what to exclude?

2

Would it work to choose different env vars for an edge handler vs functions?

3

hm, i’m not sure, and we can definitely go find out more, but will you first check if there is something useful in here?

4

Yeah I’ve read through that thread, thanks for sharing it. I didn’t find where a solution to my problem is presented. Maybe you can be more specific if you feel there is a solution there.

To put my problem more simply, the sum total of all Netlify UI env vars exceeds 4kb. >1kb needed in the react app, >1kb needed in ALL Netlify Lambda Functions, and >3kb is only needed in a SINGLE Netlify function and not needed on the front end React app at all nor any of the rest of the sites functions.

As it currently behaves, the deploy of the Netlify Functions includes ALL env vars (without magic prefixes for front end). This behavior of including all env vars in each function deploy causes the Lambda deploys to fail.

My goal is to be able to use the Netlify UI to manage all of the env vars in a single place outside of the repo. I could use env files and develop a webpack customization and a host of other solutions some I believe are outlined in the guide you linked to.

In the spirit of what makes Netlify the simplistic gem that it is, I’d like to be able to use the Netlify UI to manage all env vars. Using the Netlify UI has benefits over dealing with .env files that are git ignored and need to be shared among team members manually.

Are there any solutions to this issue that keep the Netlify UI as the single source of truth for all env vars for a site but allow us to manage them with more precision when the total gets beyond 4kb in size?

Are there any plans to make configurable which env vars from the Netlify UI are included during a functions deployment to AWS?

Are namespaces for Netlify UI env vars being considered? I think this or a configuration in the Netlify.toml to choose which env vars to include in a functions deployment would work too.

Are there any ways to gain access to this process to control the deployment steps? I looked at build plugins but they appear too early and we would not be able to control which env vars are included in the deployment of the Lambda to AWS. Correct me if I’m wrong please.

Thanks in advance and I want to say that I really like Netlify. Just trying to find a way to use it for how awesome it is without having to resort to workarounds etc. When I have to start doing workarounds and custom scripts and external env file management etc, the platform loses some of its benefit of simplicity and ease of use and creates a risk in presenting a simple facade but in actuality has hidden complexity that mirrors what we might see with a home grown CICD pipeline. In the latter, I’d rather have the pipeline owned 100% if it’s going to depart from what Netlify offers on the tin.

Rob

5

Hi Rob,

The limit at AWS is a hard limit; there is no changing it.
The way we send environment variables - all of them that are in the UI - is not something we’ll change very soon though there is an open feature request to which I shall add your voice (where we are considering improvements such as contextual namespaces, or “assigning” variables to specific functions, or allowing you to not-include some variables from the UI, or include variables from the toml file instead).

Sounds like you probably want to create a second site for that function with the huge env var. Or stick it in a file. That is the workaround that everyone else uses, if they really do need that data for a function.

So, sorry to say, I don’t know of any way to achieve your very specific goal since it has so many very specific requirements (MUST use the UI. CAN’T reduce variable size. NO workarounds). You can read our deploy script:

…but as of today customizing really isn’t a thing you can do directly. You can of course write build plugins to manipulate things to some degree, but that is even more work than the workarounds you dismiss out of hand, so…I don’t think it will interest you.

I do hear you on the beauty of the simple workflow. Simplicity is beauty! However, one of the things that has made our platform so simple in general is creating these facilities as “one size fits most”, rather than adding every option we can dream of to make it “one size fits all”. It does disclude some easier workflows, but most of them can be worked around by folks who are willing to adapt their workflows to the tool as it exists. I am not trying to shame you for your unwillingness, but as someone working in tech support, it is less useful to discuss what might be and more useful to discuss what is available today, when trying to solve problems folks have today.

7

I get the feeling that I’ve perturbed you in some way, but I hope that’s not the case. The Netlify team has built a great platform and I’m proud to use it.

I try never to dismiss anything “out of hand” and if I gave the impression in my reply that I did so here, then that was a mistake. I read the entire thread you shared, spent a couple hours trying the solutions presented there, and ultimately left feeling like this feature recommendation would still be a good one. My intent was to engage in a dialog about an optional configuration that should be, in my opinion, a welcome enhancement to the platform that doesn’t detract from the existing ease of use.

If making it so a Netlify Function can explicitly list which of the UI environment variables to include during build crosses the line of “one size fits most” then that’s understandable and I respect your position. I will get by with a work around and I appreciate you taking the time to respond. However, if Netlify ever decides to move that metaphorical line and consider adding this feature, I would be first to take advantage of it. I’d be happy to share ideas on how it could be implemented most effectively without disrupting the simple and approachable usability Netlify is known for.

8

Sorry to give that impression, Rob!

I was in no way perturbed, but I was bummed out that we couldn’t quite solve your issue on our platform, within your constraints. I also think the feature suggestion is a good one! Just not one that we’ll see in the next few weeks (or maybe ever), so I wasn’t relying on it to “solve” your problem, but was disheartened by the discovery that your requirements + what our platform makes available do not seem like a great fit today.

We will absolutely post in this thread should the situation here change so you can switch to better practices - no sense in building it for the people who asked for it if we don’t let them know about it.

Thanks for the followup & good luck!

If you’re open to some of those suggestions -

9

That sounds great, thanks again. I hope to see it one day!