Using require to include a relative module in Netlify Functions on Node

Hello, as i understand, Netlify Functions on Node cannot require relative modules and only work with modules from node_modules directory. Is there any clever way to import external files besides adding a compilation step to obtain a single file?

1 Like

Hi @matteo-rigon, our buildbot has the zip-it-and-ship-it module built in. On that repoā€™s readme file mentions the ability to zip your function with any dependencies. Let me know if that works for you.

Thanks Dennis, thatā€™s exacly what i was looking for!

1 Like

Hi, sorry must be missing something here but how exactly did you resolve this?

I have a couple of util functions in the root /functions dir with a package.json specifying their dependencies. I then import them into the functions themselves using relative path e.g. const twilioClient = require('../twilioClient'). This works fine locally with netlify dev.

On push, it builds successfully but the function errors when called with Cannot find module.

Any help appreciated

Even if I move the shared code into their own modules and specify in the package.json I get the same error:

// functions/package.json
"dependencies": {
  "apollo-client": "file:./__shared_code/apollo-client",
  "twilio-client": "file:./__shared_code/twilio-client"
}

When I run npm i in functions/ my modules are moved to the top level node_modules folder and (on netlify dev at least) I can import them to the function code with const apolloClient = require('apollo-client') but when deployed I get:

Error: Cannot find module 'apollo-client'

Iā€™ve added a SO Question about this here as seems like this must be a fairly standard scenario: serverless - How to share code between Netlify lambda functions - Stack Overflow

Iā€™m not sure I follow. You are trying to import a separate file? Do you have an example repository you can share? Can you provide the complete error message (which module is not found)? Youā€™ll need to make sure your functionā€™s file structure is as mentioned here.

Sorry yeah it wasnā€™t clear. Iā€™m really just wondering what the best way of sharing code between functions is? For example I need to check the JWT in authorization headers in all my functions, so would be nice to not have it repeated in each.

Or the instantiation of an my graphql and SMS handler clients. This code I need in all my functions but there doesnā€™t seem to be any easy way to make it available to each of them or at least I havenā€™t seen any examples.

It seems you can have a shared package.json and node_modules and that did appear to be working for me (locally at least) but the code I want to share is just fairly simple functions with a few lines in each and seems like a lot of effort to have to publish it as an npm module in order to share it.

Let me know if itā€™s still not clear what Iā€™m trying to do. Any help appreciated. Thanks

I understand. It really depends on how you are deploying your function. Are you using netlify-lambda to deploy or are you using our buildbotā€™s built-in zip-it-and-ship-it method?

Thanks, Iā€™m using Netlify zip it and ship it and my functions are each in their own folders inside <rootDir>/functions each with their own package.json and node_modules.

So would like to be able to import from files within the function/ folder to DRY the code a bit i.e. const apolloClient = require('../apolloClient') which does work locally but not when deployed.

The issue with that is that any js file in your functions folder will be treated as itā€™s own function. Perhaps you can store your shared code outside your functions folder.

Another approach would be to zip your function yourself. Hereā€™s an example on how to do it: function-deploy-test/package.json at master Ā· netlify/function-deploy-test Ā· GitHub
and
function-deploy-test/lambda/zipped-function at master Ā· netlify/function-deploy-test Ā· GitHub

Moving the files to a function outside the functions folder (ā€¦/modules) builds ok but fails at run time on calling the endpoint with:

   {

ā€œerrorTypeā€: ā€œRuntime.ImportModuleErrorā€,
ā€œerrorMessageā€: ā€œError: Cannot find module ā€˜ā€¦/ā€¦/modules/apolloClientā€™\nRequire stack:\n- /var/task/auth.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.jsā€,
ā€œtraceā€: [
ā€œRuntime.ImportModuleError: Error: Cannot find module ā€˜ā€¦/ā€¦/modules/apolloClientā€™ā€,
ā€œRequire stack:ā€,
ā€œ- /var/task/auth.jsā€,
ā€œ- /var/runtime/UserFunction.jsā€,
ā€œ- /var/runtime/index.jsā€,
" at _loadUserApp (/var/runtime/UserFunction.js:100:13)",
" at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",
" at Object. (/var/runtime/index.js:43:30)",
" at Module._compile (internal/modules/cjs/loader.js:955:30)",
" at Object.Module._extensionsā€¦js (internal/modules/cjs/loader.js:991:10)",
" at Module.load (internal/modules/cjs/loader.js:811:32)",
" at Function.Module._load (internal/modules/cjs/loader.js:723:14)",
" at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)",
" at internal/main/run_main_module.js:17:11"
]
}

Which is the same as I hit before so will try manually zipping next

1 Like

Sound good. Let me know how that goes.

Itā€™s unfortunate that relative imports arenā€™t supported with netlify functions OOTB. It would make a great feature!

2 Likes

Actually, as long as the file you are importing is inside the functionā€™s folder, relative importing should work. Can you describe how itā€™s not working for you? The issue on this thread is when you try to import a file outside the functionā€™s folder.

If the file must be inside the functionā€™s folder, I suppose itā€™s not shared code anymore. I had the same issue as matteo-rigon and I currently need to duplicate my code in each function to make sure it works properly in dev mode and after build.

I know I could zip my functions locally with netlify-lambda but Iā€™m used to the netlify dev command where I donā€™t have to think of bundling my files.

Could you confirm that shared code between functions is not supported for now?

1 Like

Just to clarify, netlify-lambda doesnā€™t actually zip your functions. It bundles your function and all its dependencies into a single file, which is what gets deployed.

Also, with regards to shared code, the code that handles local dependencies is here: https://github.com/netlify/zip-it-and-ship-it/blob/master/src/dependencies.js#L34-L70. From what I can tell, the code should find the file but since the require points to a path outside the function, when deployed, that require statement still points to ../somefile but that file is likely inside the function folder now, so the require would need to point to ./somefile instead. You could file an issue on that repository to report it as a bug since that code is open-source. :slight_smile: Maybe you could add a check to see if that file exists in the same path:

const fs = require('fs')

const path = './file.txt'
let outside = require("."+path)

  if (fs.existsSync(path)) {
    outside = require(path)
  }

If that doesnā€™t work, as another workaround, in the example repo I mentioned, there is an example on how to package up your function into a zip file. My repo does the packaging in the npm script but this can be refactored out to a shell script. During this packaging process, you can copy your shared code inside your functionā€™s folder and then zip it up manually. So, for example:

  • you configure functions/ as your functions folder
  • your functions live in the func/ folder so our buildbot doesnā€™t try to deploy your unpackaged functions
  • in your build script, you run npm i in your individual function folders inside func/ and copy your shared code as well
  • and you zip your function up and put that zip file inside functions/ using the following command zip -r functions/<function_name>.zip func/<function_name>

These steps results in your repository having the file structure you desire and itā€™s prepped so that our buildbot deploys your function with all the dependencies you want.

Iā€™m struggling with the same error. Iā€™m using neither netlify-lambda nor zip-it-and-ship-it, Iā€™m just trying to get my functions up and running with netlify-dev.

Please take a look at this repository. I have included the error in the README.md.

1 Like

I did get some success (locally & live) in putting shared modules in a local npm package:

/functions
  /utils
    package.json
    index.js
  /src
    /auth
      auth.js
    /trails
      trails.js
  package.json

Export all common modules in functions/utils/index.js and set property "main": "index.js" in functions/utils/package.json.

In functions/package.json install the module:

{
  "dependencies": {
    ...
    "utils": "file:utils"
  }
}

And import it in your functions (in functions/src/auth/auth.js): import { apolloClient, twilioClient } from "utils"

Please take a look at this repository for reference.

2 Likes

This is great! Thanks for sharing this. The main issue is making sure the require points to the correct path and this neatly resolves that issue. :+1:

I got mine working using the following structure and no need to have an extra build step for functions.

/ functions
  / shared
    settings.js
    ...
  / function1
    function1.js
  / function2
    function2.js
/ site
  ...

This allows me to require settings.js in both my Netlify functions and my web site (site uses webpack). I hoped the /shared directory could have lived at the root however this doesnā€™t seem to work without a build step for functions.

1 Like