Nuxt server API endpoint: error processing multipart/form-data

Support
1

Hi everyone

I’m using Nuxt 3.13.
Site: https://genuine-jelly-3eb6da.netlify.app/
DNS issues? No
Build problems? No

I have a form on my site, which make a request to Nuxt server API endpoint (https://nuxt.com/docs/guide/directory-structure/server). I tried to use 2 modules for parsing incoming HTML form data: Formidable and Busboy (via article). Both work great on my machine, but don’t work on Netlify environment. The result on Netlify was the same. Server API always returns:

error decoding lambda response: invalid status code returned from lambda: 0

Debugging on Netlify is not so easy, so I had to use some console.log() functions.

Busboy

import busboy from "busboy";
import path from "path";
import os from "os";

export default defineEventHandler(async (event): Promise<SubmitFormResult> => {
  let fields;
  let files;

  console.log("parse multipart data start");
  [fields, files] = await parseMultipartForm(event.node.req);
  console.log("parse multipart data end");
  return {};
});

function parseMultipartForm(req) {
  return new Promise((resolve) => {
    // we'll store all form fields inside of this
    const fields = {};
    const files = {};

    console.log("instantiate our busboy instance!");
    // let's instantiate our busboy instance!
    const bb = busboy({
      // it uses request headers
      // to extract the form boundary value (the ----WebKitFormBoundary thing)
      headers: req.headers,
    });

    // before parsing anything, we need to set up some handlers.
    // whenever busboy comes across a file ...
    bb.on("file", (name, file, info) => {
      const { filename, encoding, mimeType } = info;

      //save to tmp dir
      const saveTo = path.join(os.tmpdir(), `busboy-upload-${random()}`);
      file.pipe(fs.createWriteStream(saveTo));

      files[name] = {
        originalFilename: Buffer.from(filename, "latin1").toString("utf8"),
        encoding: encoding,
        mimetype: mimeType,
        filepath: saveTo,
      };
    });

    // whenever busboy comes across a normal field ...
    bb.on("field", (fieldName, value) => {
      // ... we write its value into `fields`.
      fields[fieldName] = value;
    });

    // once busboy is finished, we resolve the promise with the resulted fields.
    bb.on("finish", () => {
      console.log("finish processing request");
      resolve([fields, files]);
    });

    console.log("start processing our request");
    // now that all handlers are set up, we can finally start processing our request!
    req.pipe(bb);
    console.log("end processing our request");
  });
}

Netlify functions log:

Sep 4, 06:14:11 PM: INIT_START Runtime Version: nodejs:18.v33	Runtime Version ARN: arn:aws:lambda:us-east-2::runtime:b7500cbab7a13e2ee74c533d5f8cf7ca697047a4f35fad455d19cc4fa6ff8b84
Sep 4, 06:14:12 PM: 27bb4594 INFO   parse multipart data start
Sep 4, 06:14:12 PM: 27bb4594 INFO   instantiate our busboy instance!
Sep 4, 06:14:12 PM: 27bb4594 INFO   start processing our request
Sep 4, 06:14:12 PM: 27bb4594 INFO   end processing our request
Sep 4, 06:14:13 PM: 27bb4594 Duration: 230.02 ms	Memory Usage: 143 MB	Init Duration: 1333.49 ms

For some reason, Busboy never finishes processing request (there is no “finish processing request” console message).

Formidable

import formidable from "formidable";

export default defineEventHandler(async (event): Promise<SubmitFormResult> => {
  let fields;
  let files;

  //parse form data via formidable
  const form = formidable({ allowEmptyFiles: true, minFileSize: 0 }); //TODO: check empty file on client via yup
  try {
    console.log("parse multipart data start");
    [fields, files] = await form.parse(event.node.req);
    console.log("parse multipart data end");
  } catch (err) {
    console.error(err);
    return { isError: true, errorMsg: "Error parsing form data" };
  }
  return {};
});

Netlify functions log:

Sep 4, 06:41:54 PM: 255b7634 Duration: 6.54 ms	Memory Usage: 200 MB
Sep 4, 06:42:01 PM: c649de3e INFO   parse multipart data start
Sep 4, 06:42:01 PM: c649de3e Duration: 59.69 ms	Memory Usage: 201 MB

For some reason, Formidable never finishes processing request (there is no “parse multipart data end” console message).

I’ve spend whole day for that problem and I really need some help.
I found 2 similar problems on forum, but there is no answer:

2

I never got the bottom of what exactly was going on with my issue (#2 above). Despite trying a lot of things I could never successfully use formidable to parse the form data. I ended up wondering if formidable needs to be able to write to disk in some way which you can’t do with Netlify functions. I gave as couldn’t get any kind of useful error back to work out the problem and tried a lot things. In the end i just stopped using form data and went a different path and converted the files on the clientside ussing the native fileReader API to end up with the images base64 encoded. I was uploading them to Cloudinary but might not work if you’re doing something else with them.

Front end code:


// this is the class i used to create the uploadable file objects just FYI

export class UploadableFile {
  file: File;
  id: string;
  url: string;
  status: string | null;
  blob: Blob;
  constructor(file: File) {
    this.file = file;
    this.id = `${file.name}-${file.size}-${file.lastModified}-${file.type}`;
    this.url = URL.createObjectURL(file);
    this.status = null;
    this.blob = new Blob([file], { type: file.type });
  }
}

// function to sort out the images and pass to the upload endpoint. 

async function uploadImages(files: UploadableFile[]) {
    
let imageUrls: string[] = []

    return new Promise<{ data: string[], error: string }>(async (resolve, reject) => {

      try {
        if (files.length > 0) {
          for (let file of files) {
            try {
              const encodedFile = await readFile(file.file);
              const { data, error } = await useFetch("/api/upload", {
                method: "POST",
                body: { file: encodedFile },
              });
              if (data.value?.data) {
                imageUrls.push(data.value.data)
              }
            }
            catch (error) {
              reject({ data: imageUrls, error: error })
            }
            resolve({ data: imageUrls, error: '' })
          }
        }
      } catch (error) {
        // Catch recaptcha or server errors
        reject({ data: imageUrls, error: error })
      }
    })

// function to convert to base64 encoded strings

const readFile = (file: File) => {
  return new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.onload = () => {
      resolve(reader.result);
    };
    reader.onerror = reject;
    reader.readAsDataURL(file);
  });
};

Backend code e.g. server/api/upload.post.ts

import { v2 as cloudinary } from 'cloudinary';

export default defineEventHandler(async (event) => {
  // Set up
  const config = useRuntimeConfig(event);

  cloudinary.config({
    cloud_name: config.cloudinaryName,
    api_key: config.cloudinaryApiKey,
    api_secret: config.cloudinaryApiSecret
  });

  // Variable for respons
  let resError;
  let imageUrl: string = ''

  // Get the body of the request
  const body = await readBody(event);
  console.log(body)

  const { file } = body;

  try {
    const cloudiaryImg = await cloudinary.uploader.upload(file, {
      image_metadata: true,
    });
    if (!cloudiaryImg) {
      return { error: 'Error uploading image to cloudinary', data: imageUrl, success: false };
    }
    console.log('cloudiaryImg', cloudiaryImg.secure_url)
    imageUrl = cloudiaryImg.secure_url;
  } catch (error) {
    resError = error;
    console.error('error', error);
  }
  return { error: resError, data: imageUrl, success: true };
});

Not sure if that is helpful or not… I feel like parsing formData directly on Netlify is maybe too tricky but someone smarter than me could probably find a way!

1 Like
4

Hi everybody.

I’ve found the solution with busboy. At first, I found that page:

Lambda/Azure functions receive the raw JSON response without any lifecycle hooks. So in case of multipart/form data the JSON is there ready to parse (with boundaries and/or raw file contents which are usually not base64encoded if multipart but they usually are if it’s a file upload). Does this help?

After that I found a working example of code to parse multipart/form-data on AWS Lambda. I’ve noticed that author decode request body and then process it with busboy:

 const encodedBuf = Buffer.from(req.body, "base64");
 bb.end(encodedBuf); //bb - busboy

During debugging, I noticed that there is variable event.node.req.body (Buffer type) on Netlify env, but there is not on my server. This is quite strange.

So, there is my result working on Netlify:

import busboy from "busboy";
import { buffer } from "node:stream/consumers";

export default defineEventHandler(async (event): Promise<SubmitFormResult> => {
  let fields;
  let files;

  [fields, files] = await parseMultipartForm(event.node.req);
  console.log(fields, files); //IT WORKS
});

function parseMultipartForm(req) {
  return new Promise((resolve) => {
    // we'll store all form fields inside of this
    const fields = {};
    const files = {};

    // let's instantiate our busboy instance!
    const bb = busboy({
      headers: req.headers,
    });

    // before parsing anything, we need to set up some handlers.
    // whenever busboy comes across a file ...
    bb.on("file", async (name, fileStream, info) => {
      const { filename, encoding, mimeType } = info;

      files[name] = {
        originalFilename: Buffer.from(filename, "latin1").toString("utf8"),
        encoding: encoding,
        mimetype: mimeType,
        content: await buffer(fileStream),
      };
    });

    // whenever busboy comes across a normal field ...
    bb.on("field", (fieldName, value) => {
      // ... we write its value into `fields`.
      fields[fieldName] = value;
    });

    // once busboy is finished, we resolve the promise with the resulted fields.
    bb.on("finish", () => {
      resolve([fields, files]);
    });

    if (req?.body) {
      //lambda ?
      const encodedBuf = Buffer.from(req.body, "base64");
      bb.end(encodedBuf);
    } else {
      // now that all handlers are set up, we can finally start processing our request!
      req.pipe(bb);
    }
  });
}

P. S. Thank you Tim Chesney for trying to help.