Calling PDFkit from Lambda function (recent breaking changes?)

[broeker]

Back in June I had a working function with a dependency on PDFKit. I was able to make my own binaries for PDFKit, and upload them to /src/bin folder within my function like so:

/functions/
----- /pdf-gen/
----- /pdf-gen/src/
----- /pdf-gen/src/bin/
----> /pdf-gen/src/bin/libgcj.so.10
----> /pdf-gen/src/bin/pdftk
----- /pdf-gen/pdf-gen.js

Then during the build, Netflify would move those binaries automatically to:

process.env.LAMBDA_TASK_ROOT + "/src/bin"

I was then able to update both process.env.PATH and process.env.LD_LIBRARY_PATH so that my command would be available to run. This worked with success and I was able to execute:

exec("pdftk --version", context.done)

with no error, and was now able to run my function and generate a PDF in base64.

I’m back revisiting this application, and it is no longer working with the exact same code. (It continues to work locally using Netlify CLI, because I have installed PDFKit on my system.)

Now when I strip everything out and simply try to return pdftk --version I get command not found errors and when I run exec("ls -lah " + process.env.LAMBDA_TASK_ROOT + "/src/bin") I see directory not found.

Has something changed in the build process since June, or any reason why I can no longer run my command?

Thank you!

[broeker]

Haha, I thought I had already tried this a few times and it was not working, but it turns out my binaries are exactly where I put them after the build, and if I update my PATHs to include:

process.env.LAMBDA_TASK_ROOT + "/src/functions/pdf-gen/src/bin"

instead of:

process.env.LAMBDA_TASK_ROOT + '/src/bin'

I can now execute my commands again. Still need to put everything back in place and remove my test code but since I can now return pdftk --version again successfuly I think I’m good to go.

Would still welcome any replies from an official source – was this in an intentional change? Is there a better or more preferred way of including a binary?

FWIW I’m using straight deploys with a package.json directly in my function folder, and not the “zip-it” method I’ve read about but have not explored.

[jen]

Hey @broeker,
Nothing particularly conclusive or satisfying to share on this, but I wanted to let you know we spent some time on it. It looks like you may be running into something similar to this person:

Folder structure of function deployment differs

Ok, the explanation was even simpler: If you embed a library that needs to be installed into node_modules in the function, the file structure of the system is different. Here is the reproduction: The repository contains a no-node_modules.js and a with-node_modules.js function. Here is some logging output that shows their file system structure: no-node_modules.js: INFO process.env.LAMBDA_TASK_ROOT /var/task INFO # content of process.env.LAMBDA_TASK_ROOT INFO [ 'no-node_modules.js', '…

As my colleague mentions at the end of that thread, we’ll need to investigate and get this properly documented. There are many places where changes may have have definitely happened since June that could have impacted these paths:

• https://github.com/netlify/zip-it-and-ship-it/
• https://github.com/netlify/build/
• our proxy, which forwards your function request to AWS
• https://docs.aws.amazon.com/lambda/latest/dg/lambda-releases.html

I wish I had more to share but I hope this helps!