Local Serverless function does not reload because of bufferutil

Who is effected by this bug:
Blockchain developers who want to use serverless functions

when using netlify-cli’s netlify dev to test locally,
the bufferutil module causes serverless functions not to reload when the file is updated.
(Hangs on ◈ Reloading function {name}…).

this module is used heavily used in the websocket module, which in turn is used web3 libraries such as ethers.js which is how i discovered the problem.

const bufferUtil = require('bufferutil');

const handler = async function (event, context) {

};

export { handler };

bufferutil is a library used in the websocket library written in c for speed. Using only the fallback pure javascript version works. Something about the C compiled version breaks reloading.

try {
  module.exports = require('node-gyp-build')(__dirname);
} catch (e) {
  module.exports = require('./fallback');
}

Local Development Workaround:
Edit your bufferutil and utf-8-validate node_modules to not use the c++ libraries

What you are meaning by reload @vinh-perfectstorm? Netlify functions have an execution time limit of 10 seconds, and won’t (at least from my understanding and experience) reload themselves.

yes, completely forgot to mention this is in service of the serverless function reloading on save when using netlify’s local dev environment “netlify dev” command

Looks like the utf-8-validate module inside of the ‘ws’ (websocket) module also causes this problem since it appears to be using compiled libraries written in C. Updating the code to use the fallback bypasses the error

looks like these two dependencies within the ws websocket library are unique in that they’re written in C++ for speed but for whatever reason I dont understand, importing them twice prevents the local development pipeline from reloading serverless functions.

Hey there, @vinh-perfectstorm :wave:

Thanks for reaching out. My apologies, can you please let me know if you are currently encountering the bug you have described in your first post? If so, can you please share a site and reproduction so that we can see it?

Thank you!

Hi @hillary ,

i made a public repo with the minimum required to repro the problem located here

the netlifysite is here
https://capable-alpaca-870100.netlify.app/

steps to repro:

  1. run ‘netlify dev’
  2. hit the http://localhost:5173/.netlify/functions/api endpoint
  3. make a change to the api.js file and save
  4. project tries to re-load the api endpoint but never succeeds

I can’t reproduce this. I made a change to the said file:

✔ Waiting for framework port 5173. This can be configured using the 'targetPort' property in the netlify.toml

   ┌─────────────────────────────────────────────────┐
   │                                                 │
   │   ◈ Server now ready on http://localhost:8888   │
   │                                                 │
   └─────────────────────────────────────────────────┘

◈ Reloading function api...
◈ Reloaded function api

did you try hitting the API first from the browser?
(also, what sort of machine are you using? I’m on a standard x86 64bit PC)

I had not, but I did that now and it still appears to work.

I’m using MacBook Pro 2019.

I see, Thanks anyway for looking into it.

Further note. Supabase users, since it also uses websockets could also possibly run into this issue with local development on a 64bit windows machine.

Basically any library which uses websockets, which in turn use bufferutil & utf-8-validate

I am using supabase on a win64 machine and also ran into this issue.

The local dev workaround worked for me.

Hey there, @Randdalf :wave:

Thanks for reaching out! If you have a moment, please do share the solution the local devs shared with you. This would definitely help future forums members who encounter something similar.

Sure. I didn’t mean local devs as in people working with me - I’m working solo on a personal project :slightly_smiling_face:. I meant that you need to make changes to your local dev environment, as suggested in the OP.

To be extra precise, I locally commented out parts of the code in node_modules/bufferutil/index.js and node_modules/utf-8-validate/index.js

For example:

'use strict';

// COMMENTED OUT TO FIX NETLIFY FUNCTIONS NOT RELOADING: https://answers.netlify.com/t/local-serverless-function-does-not-reload-because-of-bufferutil/73131/9
// try {
//   module.exports = require('node-gyp-build')(__dirname);
// } catch (e) {
  module.exports = require('./fallback');
// }

My experience of this issue was that functions would reload the first few times but eventually stop working. I am pretty sure I left it for a very long time once and they did eventually reload, but that’s clearly a non-starter when iterating.

1 Like

Thanks so much for coming back and sharing this-- I appreciate it. Happy building :rocket:

external_node_modules for netlify.toml is supposed to fix this, but for some reason, it wouldn’t work, however, when I tried including the node_module paths, that fixed it.

# netlify.toml
[functions]
  included_files = [
    # For some reason this works to fix "bufferutil" error on
    # Netlify Function and external_node_modules does not
    "node_modules/bufferutil/**", 
    "node_modules/utf-8-validate/**"
  ]