Curl question - what implementation I use to make curl calls on Netlify?

[9ntonio]

Help!

I am trying to address a build issue in regards to the NPM package - node-libcurl - https://www.npmjs.com/package/node-libcurl

The logs below show that I am not able to build the package. I’m assuming I need to install openssl to my site.

Can anyone recommend the safest implementation of openssl into a Netlify site. Any feedback is greatly appreciated. Thanks.

4:24:24 PM: Failed to compile.
4:24:24 PM: 
4:24:24 PM: ./node_modules/node-libcurl/lib/binding/node_libcurl.node 1:0
4:24:24 PM: Module parse failed: Unexpected character '' (1:0)
4:24:24 PM: You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
4:24:24 PM: (Source code omitted for this binary file)
4:24:24 PM: > Build error occurred
4:24:24 PM: Error: > Build failed because of webpack errors
4:24:24 PM:     at /opt/build/repo/node_modules/next/dist/build/index.js:15:918
4:24:24 PM:     at async /opt/build/repo/node_modules/next/dist/build/tracer.js:1:525

[luke]

Hi, @9ntonio. I’m reading that error differently that you are.

To me the error is saying that the was a failure for webpack to compile its bundles based on a parsing error and that the error has nothing to do with a package install.

Would you please post a link to the site build logs (or post the full log output here)?

[9ntonio]

@luke thanks for your feedback. Here is the log you requested. It’s too large for this pot so I’m including a link: https://app.netlify.com/teams/9ntonio/builds/6018983abb93a20008dcb8f0

I noted openssl as I had the same error on my local machine until I added openssl via brew.

Please let me know if you need anything else.
Antonio

[luke]

Hi, @9ntonio. I see the curl dependency install failing in those logs:

4:22:16 PM: ==> Installing curl dependency: openssl@1.1
4:22:16 PM: ==> perl Makefile.PL INSTALL_BASE=/opt/buildhome/.linuxbrew/Cellar/openssl@1.1/1.1.1i/libexec
4:22:16 PM: ==> make PERL5LIB=/opt/buildhome/.linuxbrew/Cellar/openssl@1.1/1.1.1i/libexec/lib/perl5 CC=gcc-5
4:22:16 PM: ==> make install
4:22:16 PM: ==> perl Makefile.PL INSTALL_BASE=/opt/buildhome/.linuxbrew/Cellar/openssl@1.1/1.1.1i/libexec
4:22:16 PM: ==> make PERL5LIB=/opt/buildhome/.linuxbrew/Cellar/openssl@1.1/1.1.1i/libexec/lib/perl5 CC=gcc-5
4:22:16 PM: ==> make install
4:22:16 PM: ==> perl Makefile.PL INSTALL_BASE=/opt/buildhome/.linuxbrew/Cellar/openssl@1.1/1.1.1i/libexec
4:22:16 PM: ==> make PERL5LIB=/opt/buildhome/.linuxbrew/Cellar/openssl@1.1/1.1.1i/libexec/lib/perl5 CC=gcc-5
4:22:16 PM: ==> make install
4:22:16 PM: ==> perl ./Configure --prefix=/opt/buildhome/.linuxbrew/Cellar/openssl@1.1/1.1.1i --openssldir=/opt/buildhome/.linuxbrew/etc/openssl@1.1 no-ssl3 no-ssl3-method no-zlib enable-md2 linux-x86_64
4:22:16 PM: ==> make
4:22:16 PM: ==> make install MANDIR=/opt/buildhome/.linuxbrew/Cellar/openssl@1.1/1.1.1i/share/man MANSUFFIX=ssl
4:22:16 PM: ==> make test
4:22:16 PM: Last 15 lines from /opt/buildhome/.cache/Homebrew/Logs/openssl@1.1/13.make:
4:22:16 PM: ../test/recipes/99-test_ecstress.t ................. ok
4:22:16 PM: ../test/recipes/99-test_fuzz.t ..................... ok
4:22:16 PM: Test Summary Report
4:22:16 PM: -------------------
4:22:16 PM: ../test/recipes/30-test_afalg.t                  (Wstat: 256 Tests: 1 Failed: 1)
4:22:16 PM:   Failed test:  1
4:22:16 PM:   Non-zero exit status: 1
4:22:16 PM: Files=158, Tests=2635, 138 wallclock secs ( 2.12 usr  0.27 sys + 100.45 cusr 32.58 csys = 135.42 CPU)
4:22:16 PM: Result: FAIL
4:22:16 PM: Makefile:207: recipe for target '_tests' failed
4:22:16 PM: make[1]: *** [_tests] Error 1
4:22:16 PM: make[1]: Leaving directory '/tmp/openssl-1.1-20210202-18131-13cocy8/openssl-1.1.1i'
4:22:16 PM: Makefile:205: recipe for target 'tests' failed
4:22:16 PM: make: *** [tests] Error 2
4:22:16 PM: READ THIS: https://docs.brew.sh/Troubleshooting
4:22:16 PM: These open issues may also help:
4:22:16 PM: Cannot update openssl@1.1 because their test fails https://github.com/Homebrew/linuxbrew-core/issues/22015
4:22:16 PM: Installing curl has failed!
4:22:16 PM: Homebrew Bundle failed! 1 Brewfile dependency failed to install.

The logs above include this link to the open issue:

Cannot update openssl@1.1 because their test fails

So, yes, there is a bug in the Homebrew install of this package.

[9ntonio]

Thanks @luke for your help on this, I appreciate the feedback. Knowing that the build is failing via brew, can you recommend another way of installing openssl in my build?

Thanks.

[luke]

Hi, @9ntonio. I have no idea. I’d be happy to learn about this with you but I’ll need more information to answer.

First, curl and openssl should already be available in our build image so I don’t understand why these Homebrew packages are being used in the first place. Also, most site using node to build will be using npm or yarn to manage their dependencies so, again, this use of Homebrew is puzzling to me.

Have you tried removing the Homebrew dependency and using the npm package instead?

curl

[9ntonio]

@luke thanks for the info on your build images. I wasn’t aware that both packages were already installed. On my end I am using this node package - https://www.npmjs.com/package/node-libcurl which allows me to create curl calls using js promises. I will try to remove the brew package install and see if the node package errors out or builds. Either way I will post my findings. Thanks.

[9ntonio]

@luke I was able to resolve my issue via the addition of the next.config.js file. I removed any Homebrew installs and I was still getting webpack loader errors. I needed to let webpack load .node file types. I did this with the next.config.js file and the following code within it.

module.exports = {
  target: 'experimental-serverless-trace',

  // !! 'node-libcurl' needs to be able to access '.node' files.
  // !! 'node-loader' is used as the loader via this webpack config.
  webpack(config, options) {  
    config.module.rules.push({
      test: /\.node$/,
      loader: 'node-loader',
    });

    return config;
  }
};

I also needed to install the node-loader npm package to help resolve the issue. Note the loader type above node-loader. LMK if you have any questions.