Yes, it is working fine when sending from the postman but it is not working when sending the data by code. You can replicate the above linked Multipart Form Data with Netlify function post.
I think in that case the problem would be that since Forms are encoded with Multipart content header, they’re excluded from the base64 encoding of the data.
Could you try to perform base64 encoding on the client-side and send that along with the form data and see how it goes?
It is not working. Have you tried on your end?
Could you post your code example that’s not working?
Here is the netlify function code.
const Busboy = require("busboy");
const FormData = require("form-data");
const fetch = require("node-fetch");
function parseMultipartForm(event) {
// source: https://www.netlify.com/blog/2021/07/29/how-to-process-multipart-form-data-with-a-netlify-function/
return new Promise(resolve => {
const fields = {};
const busboy = new Busboy({
// uses request headers to extract the form boundary value
headers: event.headers,
});
busboy.on("file", (fieldname, filestream, filename, transferEncoding, mimeType) => {
// ... we take a look at the file's data ...
filestream.on("data", data => {
fields[fieldname] = {
filename,
type: mimeType,
content: data,
transferEncoding,
};
});
});
// whenever busboy comes across a normal field ...
busboy.on("field", (fieldName, value) => {
// ... we write its value into `fields`.
fields[fieldName] = value;
});
// once busboy is finished, we resolve the promise with the resulted fields.
busboy.on("finish", () => {
resolve(fields);
});
// now that all handlers are set up, we can finally start processing our request!
busboy.write(event.body);
});
}
exports.handler = async function(event, context) {
// parse the incoming multipart/form-data data into fields object
const fields = await parseMultipartForm(event);
// create new formdata object to be send to Lever
const form = new FormData();
for (const [key, value] of Object.entries(fields)) {
if (key === "resume") {
// append "resume" with the file buffer and add the file name
form.append("resume", value.content, { filename: value.filename });
} else {
form.append(key, value);
}
}
};
When console log the event.body, it has more size for pdf file than the original uploaded file.
Found one thread regarding multipart/form data → Netlify server functions unable to handle multipart/form-data
Hi @Coderr,
It’s because the event.body is base64 encoded.
You need to use Buffer.from(event.body, 'base64').toString('utf8').
Full code:
import { initializeApp } from 'firebase/app'
import { getStorage, ref, uploadString } from 'firebase/storage'
const Busboy = require('busboy')
exports.handler = async event => {
return new Promise(resolve => {
const fields = {}
const busboy = new Busboy({
headers: event.headers
})
busboy.on('file', (fieldname, filestream, filename, _, mimeType) => {
filestream.on('data', data => {
fields[fieldname] = {
content: data,
filename,
type: mimeType
}
})
})
busboy.on('field', (fieldName, value) => {
fields[fieldName] = value
})
busboy.on('finish', () => {
resolve(fields)
})
busboy.write(Buffer.from(event.body, 'base64').toString('utf8'))
}).then(formData => {
return uploadString(ref(getStorage(initializeApp({
/* firebase stuff */
})), formData.name), formData.file.content.toString('base64'), 'base64').then(() => {
return {
body: JSON.stringify(true),
statusCode: 200
}
})
})
}
Note: This is my form:
<form enctype = "multipart/form-data" method = "POST" name = "fileForm">
<p>
<label>
File name:
<input type = "text" name = "name"/>
</label>
</p>
<p>
<label>
File:
<input type = "file" name = "file"/>
</label>
</p>
<p>
<button>
Send
</button>
</p>
</form>
So if your form has different fields and names (it most likely will), you’d have to make adjustments in your serverless code.
After adding Buffer.from(event.body, ‘base64’).toString(‘utf8’) in busboy.write, I am getting below timed out error. Is the PDF content working on your above code?
{"errorMessage":"Task timed out after 10.00 seconds","errorType":"TimeoutError","stackTrace":["new TimeoutError (lib/node_modules/netlify-cli/node_modules/lambda-local/build/lib/utils.js:112:28)
This time I tried a simple 20 KB YML file as that was readily available on my Desktop. I was able to get the file in my Firebase Storage and it was MD5 perfect. So I assumed it would work for other files too.
How big is the PDF that you’re trying to use?
Also, I think the error you’ve got is in Netlify CLI. Netlify CLI doesn’t encode the event.body as base64 (yet - I’ll be filing as issue for that later today). So, in Netlify CLI, you have to use it like before, the above code would only work in Production.
If you wish to support both the environments, you would have to do something like:
if (event.isbase64Encoded) {
busboy.write(Buffer.from(event.body, 'base64').toString('utf8'))
} else {
busboy.write(event.body)
}
OR one liner (might not work):
busboy.write(event.isbase64Encoded ? Buffer.from(event.body, 'base64').toString('utf8') : event.body)
I will try the above. I also used simple pdf file size like 50kb.
Still pdf is corrupted. Actually by default ‘isBase64Encoded’ is coming as false when checked the console log of event.
May be this causes issue. → Netlify server functions unable to handle multipart/form-data - #4 by step
As I said, in CLI it’s always going to be false.
The PDF you’re trying to use would be useful to test if you still think it’s not working. Without that, I don’t have any way to reproduce what you’re saying.
According to the file that was sent and tested, yes there seems to be a problem in handling non-text based files. So images, PDFs or any other file type for that matter that’s not plain-text would not work. I’ll see if there’s something that can be done about this.
Yes, both images and pdf are not working. Maybe they need to include multipart/form data in the new change mentioned here → Changed behavior in function body encoding
So, I tested this and I’ve 2 options to offer:
- Continue using
multipart/form-data:
Yes, you could continue using it, but with a catch. If you only need to upload a file with the form (and no other form fields), this would do the trick. In the above code, simply change it to Buffer.from(event.body, 'base64'). I repeat, there should not be any other data except the file. Not multiple files either, just 1 file! If you need multiple files, you’d have to send each file to a different serverless function. You can use the same function, but you’d have to invoke it again for each file. For form fields, you could use FormData(), but make sure that you exclude the file from it.
- Switch to
application/json:
Yes, submitting a form with application/json might seem weird, but it works. However, there’s some extra client-side work that’s required. You need to convert the file to base64 and add each form field separately to the JSON payload. You could do it programatically too, but yeah that’s the workflow. Here’s an example:
const form = document.forms.fileForm
form.onsubmit = event => {
event.preventDefault()
new Promise((resolve, reject) => {
const reader = new FileReader()
reader.readAsDataURL(form.elements.file.files[0])
reader.onload = () => resolve(reader.result)
reader.onerror = error => reject(error)
}).then(file => {
fetch('/.netlify/functions/getData/', {
body: JSON.stringify({
file: file,
name: form.elements.name.value
}),
headers: {
'content-type': 'application/json'
},
method: 'post',
}).then(response => {
if (response.ok) {
return response.json()
} else {
throw response.statusText
}
}).then(data => {
console.log(data)
}).catch(error => {
console.log(error)
})
})
}
Finally, in the serverless function, no need to use Busboy anymore:
import { initializeApp } from 'firebase/app'
import { getDownloadURL, getStorage, ref, uploadString } from 'firebase/storage'
exports.handler = async event => {
const payload = JSON.parse(event.body)
return uploadString(ref(getStorage(initializeApp({
/* firebase stuff */
})), payload.name), payload.file.split(',')[1], 'base64').then(file => {
return getDownloadURL(file.ref).then(url => {
return {
body: JSON.stringify({
url: url
}),
statusCode: 200
}
})
})
}
Basically, the file’s base64 content exists as: payload.file.split(',')[1]. The file gets encoded as follows:
data:application/pdf;base64,actual-base64-content
And thus, we need to split it to get the actual base64 content. That’s required at least to upload the file to Firebase, it might not be required for other providers - but if you wish to convert the base64 to some other format like Buffer or something else, you’d again have to use split(',')[1].
Using the JSON way, you get both, the form fields and the file - MD5 perfect.
Test it here:
https://stoic-brown-d4563d.netlify.app/
Name is supposed to be the file name that you’d like it to be saved as.
You’d get the download URL of the file in console or if the console trims it out, you could always check it in Network tab.
Hi Folks!
I’m struggling to get a similar netlify function working. I’m trying to upload an image to Cloudflare Images I too, followed that netlify blog post, and I think I’m close to getting it working, but I’m struggling to upload an image, I get the error returned from cloudflare ERROR 9422: Decode error: image failed to be decoded: Uploaded image must have image/jpeg or image/png content type
So what I’m asking for help with, is how to convert that uploaded file buffer into a jpeg/png?
My complete code is in this gist, and I got it working easily with a plain express version (code here) – any help gratefull received!
const fields = await parseMultipartForm(event)
const data = new FormData()
// data.append('file', fields.uploadedImage.content.toString('base64'), { type: 'image/jpeg' })
data.append('file', Buffer.from(fields.uploadedImage.content, 'base64'))
await axios({
method: 'post',
url: `https://api.cloudflare.com/client/v4/accounts/${process.env.CLOUDFLARE_ACCOUNT_ID}/images/v1`,
headers: {
Authorization: `Bearer ${process.env.CLOUDFLARE_API_TOKEN}`,
'Content-Type': 'multipart/form-data; boundary=' + data._boundary,
},
data
})
.then((response) => {
return {
statusCode: 200,
body: JSON.stringify({ response }),
}
})
.catch((error) => {
console.log({ error })
return {
statusCode: 500,
body: JSON.stringify({ error }),
}
})
I can see that Cloudflare needs the content type as that.
But you seem to be sending this.
Hi, yeah it needs to be sent as that, otherwise it returbs ERROR 5415: Images must be uploaded as a form, not as raw image data. Please use multipart/form-data format
I’m sure it’s something to do with encoding the image buffer into a jpeg to attach to the formdata
In case this is any use, this is what I get from my fields when I send it an image
{
"content": {
"type": "Buffer",
"data": [239,191,189,239,191,189,239,191,189,239,191,.....etc]
},
"filename": {
"filename": "temp-join-us.jpg",
"encoding": "7bit",
"mimeType": "image/jpeg"
}
}