How to include dependencies in Netlify Lambda Functions?

[bradleygriffith]

@fool

Otherwise, functions will be shipped by just making a zip file out of the single function file or folder. functions/my-function.js or functions/my-function/handler.js .
In the second case with the subdirectory it’s possible to put a package.json into it and run cd functions/my-function && npm i at build time. (Sadly this is manual right now, but we are working on improvements)

Does this actually work right now?

My folder structure is as follows (simplified for example):

ProjectName/
  app/
  functions/
    fooFunction/
      node_modules/
      .gitignore
      fooFunction.js
      package.json  
  node_modules/
  .gitignore
  package.json

Where each .gitignore file includes the relevant node_modules/ dir, keeping third party node modules out of version control / GitHub, as one would expect.

Due to many reasons but not least of which the ongoing recommendation from Netlify, Ive migrated to using the CLI, or “Netlify dev”, but wish to continue using the continuous integration capabilities provided by Netlify, where my code is simply deployed when merged into my main (master for me) branch.

The way I understand things working is that when an “unbundled” function directory is merged or updated in my main branch, that Netlify will automatically run a compression process (Zip) on it, and then deploy the zipped file to Lambda. Given the advice above, it seems that the general method we are supposed to use to get our node modules included into this process without having to commit them outright, is to have a script prepended to our main build script, that first installs node modules in our functions. To that end my main root-level (app) package.json has the step, buildFunctions prepended to it (build is the script Ive told Netlify to run during deployment):

  "scripts": {
    "buildFunctions": "cd functions/fooFunction && npm install && cd ../../",
    "build": "npm run buildFunctions && webpack --config webpack.app.production.js",
    ...
  }

During deployment, I can actually see this being ran in the Netlify console. First it changes directories into my fooFunction folder, then it installs modules - which I can see the output for -, then it goes back to root and runs the rest of the build, deploying my site.

Here is a slightly edited bit of my deploy output showing this stuff working:

... stuff ...
2:23:09 AM: Executing user command: npm run build
2:23:10 AM: > ProjectName@0.0.1 build /opt/build/repo
2:23:10 AM: > npm run buildFunctions && webpack --config webpack.app.production.js
2:23:10 AM: > ProjectName@0.0.1 buildFunctions /opt/build/repo
2:23:10 AM: > cd functions/fooFunction && npm install && cd ../../
2:23:14 AM: added 37 packages from 53 contributors and audited 56 packages in 2.902s
2:24:34 AM: Hash: abcdefghijklmnop
2:24:34 AM: Version: webpack 4.42.1
2:24:34 AM: Time: 78343ms
2:24:34 AM: Built at: 04/20/2020 6:24:34 AM
... a little later ...
2:24:34 AM: Function Dir: /opt/build/repo/functions
2:24:34 AM: TempDir: /tmp/zisi-abcdefghijklmnop
2:24:35 AM: Prepping functions with zip-it-and-ship-it 0.3.1
2:24:36 AM: [ { path: '/tmp/zisi-abcdefghijklmnop/fooFunction.zip',
2:24:36 AM:     runtime: 'js' } ]
2:24:36 AM: Prepping functions complete
... etc ...

However, after deployment, if I try to run my function it errors with a 502 (aside, can someone point me to documentation about why Netlify uses 502 status for these errors? Id like to catch in frontend and be confident that this is always going to be the status of a function error). If I inspect the output of the console for the function, I see this:

10:58:05 AM: 2020-04-20T14:58:05.146Z undefined ERROR Uncaught Exception  {"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module 'googleapis'\nRequire stack:\n- /var/task/lib/services/GoogleSheets/SheetService.js\n- /var/task/fooFunction.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js","stack":["Runtime.ImportModuleError: Error: Cannot find module 'googleapis'","Require stack:","- /var/task/lib/services/GoogleSheets/SheetService.js","- /var/task/fooFunction.js","- /var/runtime/UserFunction.js","- /var/runtime/index.js","    at _loadUserApp (/var/runtime/UserFunction.js:100:13)","    at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)","    at Object.<anonymous> (/var/runtime/index.js:43:30)","    at Module._compile (internal/modules/cjs/loader.js:1158:30)","    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1178:10)","    at Module.load (internal/modules/cjs/loader.js:1002:32)","    at Function.Module._load (internal/modules/cjs/loader.js:901:14)","    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)","    at internal/main/run_main_module.js:18:47"]}

Which appears to imply that the node_modules were never zipped into the function. That is, why cant it find the module, googleapis (Ive noticed many others having issues with googleapis… is there a problem with this package specifically?).

Here is what’s in my function’s package.json:

  "dependencies": {
    "googleapis": "^25.0.0"
  },

**** Also want to add: I know there is other advice to just add the function dependencies to the root-level package.json. Id very much like not to do this. Im trying to build a long-term maintainable system, and anticipate the possibilities of growing function needs (requiring more and more additions) AND/OR of wanting to untie my AWS Lambda function usage from being managed by Netlify (I already have a secondary functions folder for this because I have non-HTTP “server” specific Lambda needs that Netlify isnt built to help with), and the idea of complicating my app’s package.json dependencies for this setup doesnt feel like a wise decision.