Module mysteriously not found error - same import works elsewhere

Site name: beamish-halva-baf90b

DNS issues: No.

Build problems: No. I wish that errors in importing modules created build issues - Then I could know what was going wrong here.

Did you try Ask Netlify: Yes, and it was miserable. 300 char limit and the inability to eg press shift+enter to add a newline made it difficult to give any context or info. But, the ultimate result was “That’s quite perplexing”.

Short Version

I have two handlers across two scripts for Netlify Functions. There is one which works, including the exact same import statements and more, and a second which includes a copy/pasted subset of those imports which errors with ERR_MODULE_NOT_FOUND - for the exact same module that was found elsewhere! You can see the imported function + module here.

And I need to emphasize here that loading the exact same module with the exact same config sometimes works, and sometimes results in the module not being found, for no discernible reason.

Additional Context

createHandler catches and handles any thrown errors, and returns something via Response.json. Any errors must be in the static imports. There are not dynamic imports via import() or anything here. And the response given from this endpoint it “error decoding lambda response: error decoding lambda response: unexpected end of JSON input”.

The error from logs

The important part is “Cannot find package ‘@shgysk8zer0/jwk-utils’ imported from /var/task/node_modules/@shgysk8zer0/lambda-http/handler.js”. I import literally the exact same thing and more in another handler without issue, but for seemingly no reason the same import (literally copy/pasted) does not work in a much simpler case.

Sep 21, 08:38:22 AM: 5d821661 ERROR  2024-09-21T15:38:22.911Z	undefined	ERROR	Uncaught Exception 	{
	"errorType": "Error",
	"errorMessage": "Cannot find package '@shgysk8zer0/jwk-utils' imported from /var/task/node_modules/@shgysk8zer0/lambda-http/handler.js",
	"code": "ERR_MODULE_NOT_FOUND",
	"stack": [
		"Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@shgysk8zer0/jwk-utils' imported from /var/task/node_modules/@shgysk8zer0/lambda-http/handler.js",
		"    at packageResolve (node:internal/modules/esm/resolve:858:9)",
		"    at moduleResolve (node:internal/modules/esm/resolve:931:18)",
		"    at moduleResolveWithNodePath (node:internal/modules/esm/resolve:1173:14)",
		"    at defaultResolve (node:internal/modules/esm/resolve:1216:79)",
		"    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:540:12)",
		"    at ModuleLoader.resolve (node:internal/modules/esm/loader:509:25)",
		"    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:239:38)",
		"    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:96:40)",
		"    at link (node:internal/modules/esm/module_job:95:36)"
	]
}

@shgysk8zer0/jwk-utils is imported as a dependency of @shgysk8zer0/lamdba-http. Both indirectly import @shgysk8zer0/jwk-utils via import { createHandler } from '@shgysk8zer0/lambda-http'.

The config via netlify.toml

[build]
  base = "./"
  publish = "./_site"
  command = "npm run build && npm run build:site"
  functions = "api"
[dev]
  base = "./"
  publish = "./_site"
  command = "npm run serve:dev"
  functions = "api"
  targetPort = 8080
[functions]
  external_node_modules = [
    "node:fs",
    "node:path",
    "node:fs/promises",
    "node:process",
    "node:url",
    "node:perf_hooks",
    "node:util",
    "node:test",
    "node:assert",
    "node:os",
    "node:child_process",
    "node:events",
    "node:crypto"
  ]
  node_bundler = "esbuild"
  included_files = ["_data/jwk.json"]

Note that there is no script-specific config here. It’s not like there is anything different in module bundling or config/externals.

I have also tried including/excluding various packages from external_node_modules and it made no difference.

In my more recent efforts to figure out what’s going on here, I attempted to create a function, using only imports that work elsewhere, that would just return the resulting code.

import { createHandler } from '@shgysk8zer0/lambda-http';
import { readFile } from 'node:fs/promises';

export default createHandler({
	async get() {
		try {
			const path = new URL(import.meta.url)?.pathname;
			const content = await readFile(path, { encoding: 'utf-8' });
			return new Response([content], { headers: { 'Content-Type': 'text/plain', 'X-Path': path }});
		} catch(err) {
			console.error(err);
			return Response.json(err, { status: 500 });
		}
	}
});

@shgysk8zer0/lambda-http and node:fs/promises are all successfully imported/found elsewhere.

I’ve also tried even the most basic version (with error):

/* eslint-env node */
import { createHandler } from '@shgysk8zer0/lambda-http';

export default createHandler({
	async get() {
		return new Response(['Hello, World!'], { headers: { 'Content-Type': 'text/plain' }});
	}
});

And also a better testing version to check the output (which works fine):

import { readFile } from 'node:fs/promises'

export default async () => {
	const url = new URL(import.meta.url);
	const content = await readFile(url.pathname, { encoding: 'utf-8' });
	return new Response([content], { headers: { 'Content-Type': 'text/plain' }});
};

Since logs require a production build, I cannot say it is the same error, but… The error is pretty obviously in import { createHandler } from '@shgysk8zer0/lambda-http.

Also, in the versions of echoing script/handler output, I can verify that my local and working version is perfectly identical to what is run in at least a PR build, with the exception of local code including a source map. I copied the response text of the version that reads from import.meta.url pathname, and it was identical to my local .netlify/functions-serve/ version, aside from the import map.

And, to repeat the confusing part here… import { createHandler } from '@shgysk8zer0/lambda-http' works without any issue whatsoever in other handlers. Even handlers that actually do make use of @shgysk8zer0/jwk-utils (the module not found reported in logs). It’s a static import and most definitely does exist and can be found.

Why could I possibly be getting this error sometimes but not others… Same imports, same config… It makes zero sense.

Not sure about the intermittent issues but you could use:

export const handler = createHandler()

instead of export default along with node_bundler = "esbuild" in netlify.toml like:

[functions]
  node_bundler = "esbuild"

However, probably one of the most important features of that library (Request/Response) is not built-into Netlify Functions, so you might not even need it.

Isn’t the difference between export const handler and export default that the default export (in node 18 or 20, I think) that it does work with Request or Response objects instead of some request event object? It’s shown right away at Netlify Functions using a default export with Request and Response.

The purpose of createHandler is to fix a couple of mistakes in the Request object given (set the correct referrer and such via the headers), do some request filtering such as rejecting requests missing required headers with an appropriate status code, deal with a lot of CORS issues, and to simplify working with different HTTP methods. It just makes things a lot easier to write and avoids a lot of duplicate code.

that it does work with Request or Response objects instead of some request event object

Yes, does the library expect event or request? I thought the goal of that library was to convert event to request, but maybe I was wrong.

If you need a simpler way to handle methods and routing, you can take a look at Hono: Hono - Web framework built on Web Standards. You can use:

export default async function(req, context) {
  const app = new Hono()
  return app.fetch(req, {
    context
  })
}

In any case, if you absolutely want to stay with that library and get it working, you might want to check out: fix: decodeRequestToken not bundled by hrishikesh-k · Pull Request #57 · shgysk8zer0/lambda-http (github.com)

Wow… Wasn’t expecting a PR on one of may packages. Thanks!

I asked in the comments on the PR, but maybe this is a better channel. Could you explain the difference in how the bundler works? Does it not fully support exports?

I’ve replied there. I assume that resovled the issue?