I might be going mad over here, so better ask here:
I have a lambda function foo that serves a file from the same directory as the function file. Upon deployment the file sometimes seems to live right next to the function Javascript file (whose folder path I can get via process.env.LAMBDA_TASK_ROOT), sometimes in ./src, sometimes in ./src/foo or even ./src/functions/foo.
INFO process.env.LAMBDA_TASK_ROOT /var/task
INFO # content of process.env.LAMBDA_TASK_ROOT
INFO [ 'image.js', 'src' ]
INFO # content of process.env.LAMBDA_TASK_ROOT/src
INFO [ 'image.js', 'logo.png' ]
INFO # content of process.env.LAMBDA_TASK_ROOT/src/functions
INFO undefined
INFO # content of process.env.LAMBDA_TASK_ROOT/src/functions/image
INFO undefined
And this is the other deployment with pretty much the same code (same folder structure etc), but totally different folder structure on the deployment:
INFO process.env.LAMBDA_TASK_ROOT /var/task
INFO # content of process.env.LAMBDA_TASK_ROOT
INFO [ 'image.js', 'src' ]
INFO # content of process.env.LAMBDA_TASK_ROOT/src
INFO [ 'functions', 'node_modules' ]
INFO # content of process.env.LAMBDA_TASK_ROOT/src/functions
INFO [ 'image' ]
INFO # content of process.env.LAMBDA_TASK_ROOT/src/functions/image
INFO [ 'image.js', 'logo.png' ]
Locally with netlify dev the files are directly in process.env.LAMBDA_TASK_ROOT.
Any idea what is going on here?
I am a bit scared that I am doing something fundamentally wrong or the deployments will change their behavior and break my site
Hi! I’ve not tested this but I think it all depends on how you deploy your function. Differences between single-file function (like what you have), function in a folder, and self-zipped. So it makes sense that image file is in the same folder as your single-file function. If it was a function-in-a-folder style deploy, then the image would be inside the folder, like in my example.
Unfortunately not, as they are both pretty much identical. Both deployments have 1 function in a subfolder of functions - and their deployment structure is different. (Happy to share the actual deployment via private message if you want to look into them and the difference)
Do you have example sites that you can share out in the open that shows the difference where the same function was behaving differently? Note that netlify-lambda will work differently from a deployed function as it’s mainly used to test general function logic rather than emulating every minutiae.
You can compare https://netlify-functions-image-foo.netlify.app/logo.png to https://images2.prisma.io/footer-logo.png. Under the hood they both execute a function functions/image/image.js that tries to server a file from the same folder. To make it work in both cases I needed to use a different path - see the log output in my original message. (The sourcecode of the second one is unfortunately not public on Github, but I hope you can access the code of the function on the deployment.)
The only thing I can think of is that the deploys were built using different versions of our build-image/buildbot. Could you provide links to specific deploys? The deploy should say what version is used at the beginning of the logs (at the top).
Additionally, the way functions are bundled is determined by the zip-it-and-ship-it package here: GitHub - netlify/zip-it-and-ship-it: Intelligently prepare Node.js Lambda functions for deployment which is what we use in the buildbot. If you provide those specific deploy links, we can also confirm if there was a change in the way a function is bundled that occurred between the two deploys.
❯ Loading plugins
- netlify-plugin-debug-cache@1.0.3 from Netlify app
...
┌──────────────────────────────────────────────────┐
│ 3. onEnd command from netlify-plugin-debug-cache │
└──────────────────────────────────────────────────┘
Saving cache file manifest for debugging...
Cache file count: 0
Cache manifest saved to /opt/build/repo/cache-output.json
Please download the build files to inspect cache-output.json.
Instructions => http://bit.ly/netlify-dl-cache
(netlify-plugin-debug-cache onEnd completed in 214ms)
But I added that while already debugging what was wrong.
Ah, and one has Starting to deploy site from '' and the other one Starting to deploy site from '/'. Not really sure where that difference comes from. They are configured the same way in the accounts.
The publish directory difference shouldn’t affect how a function is bundled and deployed. Since I can’t see the repository for ‘images2-prisma-io’, it’s hard to say why there is a difference. Like you said, those examples have different ‘deployment structures’.
One thing to note is that my zipped-function example is based on a function that was bundled manually (not by the buildbot’s zip-it-and-ship-it feature). That’s probably the best option to take that would give you a consistent path for your bundled files/images.
sorry to hear that you are still having difficulties with this.
Unfortunately, with only seven support engineers who provide support for hundreds of thousands of people worldwide, we can’t, unfortunately, go more in depth here than we already have - getting involved in individual git repo’s and exchanging information via DM is outside of the scope of support we can readily priovide.
If you can create a publicly shareable git repo with some dummy code, if the issue is very sensitive, you can post it here and maybe one someone will have more ideas for you!
As soon as I have the line const { PrismaClient } = require('@prisma/client') in my function, the path of the images change. Will create a super simple reproduction with that.
Ok, the explanation was even simpler:
If you embed a library that needs to be installed into node_modules in the function, the file structure of the system is different.
Here is the reproduction:
The repository contains a no-node_modules.js and a with-node_modules.js function.
Here is some logging output that shows their file system structure:
no-node_modules.js:
INFO process.env.LAMBDA_TASK_ROOT /var/task
INFO # content of process.env.LAMBDA_TASK_ROOT
INFO [ 'no-node_modules.js', 'src' ]
INFO # content of process.env.LAMBDA_TASK_ROOT/src
INFO [ 'functions', 'node_modules' ]
INFO # content of process.env.LAMBDA_TASK_ROOT/src/functions
INFO [ 'no-node_modules' ]
INFO # content of process.env.LAMBDA_TASK_ROOT/src/functions/no-node_modules
INFO [ 'netlify.png', 'no-node_modules.js' ]
with-node_modules.js:
INFO process.env.LAMBDA_TASK_ROOT /var/task
INFO # content of process.env.LAMBDA_TASK_ROOT
INFO [ 'src', 'with-node_modules.js' ]
INFO # content of process.env.LAMBDA_TASK_ROOT/src
INFO [ 'netlify.png', 'with-node_modules.js' ]
INFO # content of process.env.LAMBDA_TASK_ROOT/src/functions
INFO undefined
INFO # content of process.env.LAMBDA_TASK_ROOT/src/functions/with-node_modules
INFO undefined
Different although the function code is almost identical but that one line.
So he answer to my initial question is:
As soon as you use a library that needs a node_modules folder in the function deployment, you will have a different file system than is documented and mentioned in most Netlify Community posts.
I am not sure this is intended on Netlify side. If it is, it should probably be documented in some way.
