Home
Support Forums

Trouble with handling files in Netlify Function

That’s actually what I tried to achieve with Netlify. But I have now changed the logik to only use the Netlify function to get the signed URL from AWS S3 and then directly upload the image to S3 from the client…

No, it didn’t solve the issue.

I am also facing the same exact issue mentioned in the question. Followed this → How to Parse Multipart Form Data with Netlify Functions. The PDF file comes as empty when using netlify function. Txt files work fine. There is a problem with Netlify Function. The file data automatically gets corrupted. Can you check and give the fix for it?

Hi @Coderr,

I am not able to reproduce this issue. Till now, I had not tested it myself but now I did. I uploaded a PDF file via Postman to a serverless function I created and stored the file in Firebase and it worked fine.

As you can see, the size is exactly the same.

I even got the same MD5 hash for both the files (the original and the one I downloaded from Firebase). Here’s my code:

import { initializeApp } from 'firebase/app'
import { getStorage, ref, uploadString } from 'firebase/storage'

exports.handler = async event => {
  if (event.isBase64Encoded) {
    const storage = getStorage(initializeApp({
      /* firebase stuff */
    }))
    const somePDF = ref(storage, 'file.pdf')
    return uploadString(somePDF, event.body, 'base64').then(() => {
      return {
        body: JSON.stringify('Uploaded successfully'),
        statusCode: 200
      }
    })
  }
}

I simply sent a POST request to this endpoint with Postman. If you still think it’s not working for you, could you help us with an example PDF that’s not working or a reproduction case to test?

1 Like

Yes, it is working fine when sending from the postman but it is not working when sending the data by code. You can replicate the above linked Multipart Form Data with Netlify function post.

I think in that case the problem would be that since Forms are encoded with Multipart content header, they’re excluded from the base64 encoding of the data.

Could you try to perform base64 encoding on the client-side and send that along with the form data and see how it goes?

It is not working. Have you tried on your end?

Could you post your code example that’s not working?

Here is the netlify function code.

const Busboy = require("busboy");
const FormData = require("form-data");
const fetch = require("node-fetch");

function parseMultipartForm(event) {
  // source: https://www.netlify.com/blog/2021/07/29/how-to-process-multipart-form-data-with-a-netlify-function/
  return new Promise(resolve => {
    const fields = {};

    const busboy = new Busboy({
      // uses request headers to extract the form boundary value
      headers: event.headers,
    });

    busboy.on("file", (fieldname, filestream, filename, transferEncoding, mimeType) => {
      // ... we take a look at the file's data ...
      filestream.on("data", data => {
        fields[fieldname] = {
          filename,
          type: mimeType,
          content: data,
          transferEncoding,
        };
      });
    });

    // whenever busboy comes across a normal field ...
    busboy.on("field", (fieldName, value) => {
      // ... we write its value into `fields`.
      fields[fieldName] = value;
    });

    // once busboy is finished, we resolve the promise with the resulted fields.
    busboy.on("finish", () => {
      resolve(fields);
    });

    // now that all handlers are set up, we can finally start processing our request!
    busboy.write(event.body);
  });
}

exports.handler = async function(event, context) {
  // parse the incoming multipart/form-data data into fields object
  const fields = await parseMultipartForm(event);

  // create new formdata object to be send to Lever
  const form = new FormData();

  for (const [key, value] of Object.entries(fields)) {
    if (key === "resume") {
      // append "resume" with the file buffer and add the file name
      form.append("resume", value.content, { filename: value.filename });
    } else {
      form.append(key, value);
    }
  }
};

When console log the event.body, it has more size for pdf file than the original uploaded file.

Found one thread regarding multipart/form data → Netlify server functions unable to handle multipart/form-data

Hi @Coderr,

It’s because the event.body is base64 encoded.

You need to use Buffer.from(event.body, 'base64').toString('utf8').

Full code:

import { initializeApp } from 'firebase/app'
import { getStorage, ref, uploadString } from 'firebase/storage'

const Busboy = require('busboy')

exports.handler = async event => {

  return new Promise(resolve => {

    const fields = {}

    const busboy = new Busboy({
      headers: event.headers
    })

    busboy.on('file', (fieldname, filestream, filename, _, mimeType) => {
      filestream.on('data', data => {
        fields[fieldname] = {
          content: data,
          filename,
          type: mimeType
        }
      })
    })

    busboy.on('field', (fieldName, value) => {
      fields[fieldName] = value
    })

    busboy.on('finish', () => {
      resolve(fields)
    })

    busboy.write(Buffer.from(event.body, 'base64').toString('utf8'))

  }).then(formData => {

    return uploadString(ref(getStorage(initializeApp({
      /* firebase stuff */
    })), formData.name), formData.file.content.toString('base64'), 'base64').then(() => {

      return {
        body: JSON.stringify(true),
        statusCode: 200
      }

    })

  })

}

Note: This is my form:

<form enctype = "multipart/form-data" method = "POST" name = "fileForm">
  <p>
    <label>
      File name:
      <input type = "text" name = "name"/>
    </label>
  </p>
  <p>
    <label>
      File:
      <input type = "file" name = "file"/>
    </label>
  </p>
  <p>
    <button>
      Send
    </button>
  </p>
</form>

So if your form has different fields and names (it most likely will), you’d have to make adjustments in your serverless code.

After adding Buffer.from(event.body, ‘base64’).toString(‘utf8’) in busboy.write, I am getting below timed out error. Is the PDF content working on your above code?

{"errorMessage":"Task timed out after 10.00 seconds","errorType":"TimeoutError","stackTrace":["new TimeoutError (lib/node_modules/netlify-cli/node_modules/lambda-local/build/lib/utils.js:112:28)

This time I tried a simple 20 KB YML file as that was readily available on my Desktop. I was able to get the file in my Firebase Storage and it was MD5 perfect. So I assumed it would work for other files too.

How big is the PDF that you’re trying to use?

Also, I think the error you’ve got is in Netlify CLI. Netlify CLI doesn’t encode the event.body as base64 (yet - I’ll be filing as issue for that later today). So, in Netlify CLI, you have to use it like before, the above code would only work in Production.

If you wish to support both the environments, you would have to do something like:

if (event.isbase64Encoded) {
  busboy.write(Buffer.from(event.body, 'base64').toString('utf8'))
} else {
  busboy.write(event.body)
}

OR one liner (might not work):

busboy.write(event.isbase64Encoded ? Buffer.from(event.body, 'base64').toString('utf8') : event.body)

I will try the above. I also used simple pdf file size like 50kb.

Still pdf is corrupted. Actually by default ‘isBase64Encoded’ is coming as false when checked the console log of event.

May be this causes issue. → Netlify server functions unable to handle multipart/form-data - #4 by step

As I said, in CLI it’s always going to be false.

The PDF you’re trying to use would be useful to test if you still think it’s not working. Without that, I don’t have any way to reproduce what you’re saying.

According to the file that was sent and tested, yes there seems to be a problem in handling non-text based files. So images, PDFs or any other file type for that matter that’s not plain-text would not work. I’ll see if there’s something that can be done about this.

Yes, both images and pdf are not working. Maybe they need to include multipart/form data in the new change mentioned here → Changed behavior in function body encoding

So, I tested this and I’ve 2 options to offer:

  1. Continue using multipart/form-data:

Yes, you could continue using it, but with a catch. If you only need to upload a file with the form (and no other form fields), this would do the trick. In the above code, simply change it to Buffer.from(event.body, 'base64'). I repeat, there should not be any other data except the file. Not multiple files either, just 1 file! If you need multiple files, you’d have to send each file to a different serverless function. You can use the same function, but you’d have to invoke it again for each file. For form fields, you could use FormData(), but make sure that you exclude the file from it.

  1. Switch to application/json:

Yes, submitting a form with application/json might seem weird, but it works. However, there’s some extra client-side work that’s required. You need to convert the file to base64 and add each form field separately to the JSON payload. You could do it programatically too, but yeah that’s the workflow. Here’s an example:

const form = document.forms.fileForm
form.onsubmit = event => {
  event.preventDefault()
  new Promise((resolve, reject) => {
    const reader = new FileReader()
    reader.readAsDataURL(form.elements.file.files[0])
    reader.onload = () => resolve(reader.result)
    reader.onerror = error => reject(error)
  }).then(file => {
    fetch('/.netlify/functions/getData/', {
      body: JSON.stringify({
        file: file,
        name: form.elements.name.value
      }),
      headers: {
        'content-type': 'application/json'
      },
      method: 'post',
    }).then(response => {
      if (response.ok) {
        return response.json()
      } else {
        throw response.statusText
      }
    }).then(data => {
      console.log(data)
    }).catch(error => {
      console.log(error)
    })
  })
}

Finally, in the serverless function, no need to use Busboy anymore:

import { initializeApp } from 'firebase/app'
import { getDownloadURL, getStorage, ref, uploadString } from 'firebase/storage'

exports.handler = async event => {

  const payload = JSON.parse(event.body)

  return uploadString(ref(getStorage(initializeApp({
    /* firebase stuff */
  })), payload.name), payload.file.split(',')[1], 'base64').then(file => {

    return getDownloadURL(file.ref).then(url => {

      return {
        body: JSON.stringify({
          url: url
        }),
        statusCode: 200
      }

    })

  })

}

Basically, the file’s base64 content exists as: payload.file.split(',')[1]. The file gets encoded as follows:

data:application/pdf;base64,actual-base64-content

And thus, we need to split it to get the actual base64 content. That’s required at least to upload the file to Firebase, it might not be required for other providers - but if you wish to convert the base64 to some other format like Buffer or something else, you’d again have to use split(',')[1].

Using the JSON way, you get both, the form fields and the file - MD5 perfect.

Test it here:

https://stoic-brown-d4563d.netlify.app/

Name is supposed to be the file name that you’d like it to be saved as.

You’d get the download URL of the file in console or if the console trims it out, you could always check it in Network tab.