502 response from Sharp image resize function, no errors in log

Affected site: flamboyant-payne-83a356
Affected environment: https://deploy-preview-1–flamboyant-payne-83a356.netlify.app
Affected function: https://deploy-preview-1--flamboyant-payne-83a356.netlify.app/.netlify/functions/getIgImage

Greetings all!

I’ve been working on optimising a couple of features on our agency website, and have been focussing in particular on improving our Instagram feed. The Instagram Basic Display API only provides urls for large images, and we are only displaying them at 270px - That’s an awful lot of useless data we are sending to end users. On top of being bad, things like Lighthouse tend to have a bit of a problem with that.

After a couple of days and a lot of head-scratching, I managed to get Sharp up and running (well, at least I did locally) as a part of a lambda function to proxy these image requests and resize them on the fly. I had run into a bunch of issues with Sharp not working, but got around them in the end. I now have an implementation which is properly returning the resized images locally, and the function log on Netlify shows no errors (in fact, it shows the buffer that it is meant to be returning, which I would see as a good sign). However, in-browser, these requests are being returned as 502s.

There doesn’t seem to be any issue with timeouts, or anything else I can see externally or from looking at the function logs. If anyone would be able to look into this and give me a steer on what might be causing this to fall over silently, I would really appreciate it.

Single-request excerpt from the function log included for reference:

2:58:29 PM: 2020-06-03T13:58:29.219Z	c1f88c61-3e1b-49d1-b68b-b9d1270c7c1a	INFO	<Buffer ff d8 ff db 00 43 00 04 03 03 04 03 03 04 04 04 04 05 05 04 05 07 0b 07 07 06 06 07 0e 0a 0a 08 0b 10 0e 11 11 10 0e 10 0f 12 14 1a 16 12 13 18 13 0f ... 15047 more bytes>
2:58:29 PM: Duration: 109.47 ms	Memory Usage: 101 MB	Init Duration: 414.55 ms

Cheers,

David

Hi David :wave:t2:

Do you have any code you could share with this? One thing that always jumps out at me with local vs. ‘prod’ fails when it comes to functions is ensuring that your function has its own node dependencies and that your build step resolves those correctly, but it’ll be super tough to be more specific without seeing some of the bits of code that might be causing trouble… :confused:


Jon

Sure thing, @jonsully - although, as I said, it appears from the logs that everything up to the fact that the callback body seems to be empty is working fine. Here is the function in its entirety:

    import axios from 'axios'
    import sharp from 'sharp'
    
    const images = {}
    
    exports.handler = function (event, context, callback) {
      if (!event.queryStringParameters.img) {
        return callback(null, {
          statusCode: 500,
          body: 'No image requested!',
        })
      }
      const url = decodeURIComponent(event.queryStringParameters.img)
      if (images[event.queryStringParameters.img]) {
        return callback(null, {
          statusCode: 200,
          body: images[event.queryStringParameters.img],
        })
      }
    
      axios
        .get(url, {
          responseType: 'arraybuffer',
        })
        .then(response => Buffer.from(response.data, 'binary'))
        .then(img => {
          return sharp(img)
            .resize(250, 250, {
              fit: 'cover',
            })
            .jpeg({ quality: 86 })
            .toBuffer()
            .then(img => {
              console.log(img)
              images[event.queryStringParameters.img] = img
              callback(null, {
                statusCode: 200,
                body: img,
              })
            })
            .catch(err => {
              throw new Error(err)
            })
        })
        .catch(err => {
          console.log(err)
          callback(null, {
            statusCode: 500,
            body: JSON.stringify(err),
          })
        })
    }

Axios is being used in another function in the same project and is working fine there, so that isn’t a potential issue. Sharp was previously giving me issues, but seeing as I can see the buffer being created in the logs, as posted above, that does not seem like a likely issue here, either.

I have checked, just to be sure, that a Buffer is being sent to the body of the callback - By passing Buffer.from(img) rather than just img, but that makes no difference, as I suspected.

Its a puzzler to me. Everything seems fine apart from the result actually being passed back to the browser properly.

Cheers,

David

Gotcha. Yeah I think I’m getting your flow here.

That’s tricky. A 502 on lambda can mean a few different things unfortunately, but my gut feel is that it’s something with using synchronous functions && a callback(null, {thing}) rather than an async handler. Have you played with async function handlers?


Jon

I was really hoping that your gut feel would be right on this, but it appears not. I refactored the handler to be fully async and redeployed:

import axios from 'axios'
import sharp from 'sharp'

const images = {}

exports.handler = async function (event) {
  if (!event.queryStringParameters.img) {
    return {
      statusCode: 500,
      body: 'No image requested!',
    }
  }

  if (images[event.queryStringParameters.img]) {
    return {
      statusCode: 200,
      body: images[event.queryStringParameters.img],
    }
  }

  try {
    const url = decodeURIComponent(event.queryStringParameters.img)
    const response = await axios.get(url, { responseType: 'arraybuffer' })
    const imgBuffer = Buffer.from(response.data)
    const sharpImage = await sharp(imgBuffer)
    const resized = await sharpImage.resize(270, 270, {
      fit: 'cover',
    })
    const compressed = await resized.jpeg({ quality: 86 })
    const img = await compressed.toBuffer()
    console.log(img)
    images[event.queryStringParameters.img] = img
    return {
      statusCode: 200,
      body: img,
    }
  } catch (err) {
    return {
      statusCode: 500,
      body: JSON.stringify(err),
    }
  }
}

Sadly, still no dice. Once again, that console.log() logs out the buffer, but something goes wrong in actually returning it to the browser. Do I somehow need to set a header on the return setting the content type to make it work, perhaps? This isn’t my first time dealing with lambdas, but it is my first shot at using them to deal with binary data like this.

Not that it matters, because its broadly the same as earlier, but here is another sample from the function log:

11:46:38 PM: 2020-06-03T22:46:38.220Z	fc6ed781-8471-46c4-9b05-c943df1bac23	INFO	<Buffer ff d8 ff db 00 43 00 04 03 03 04 03 03 04 04 04 04 05 05 04 05 07 0b 07 07 06 06 07 0e 0a 0a 08 0b 10 0e 11 11 10 0e 10 0f 12 14 1a 16 12 13 18 13 0f ... 2327 more bytes>
11:46:38 PM: Duration: 37.74 ms	Memory Usage: 110 MB

Cheers,

D

Well (IMO) it’s certainly easier on the eyes! :stuck_out_tongue_winking_eye: I appreciate you taking the time to do that refactor. Let’s get to the bottom of this thing :smile:


I will say, the other thing that stood out to me is the

const images = {}

and

images[event.queryStringParameters.img] = img

Lambda functions have to be considered as fully stateless and stand-alone. No object from one run will ever be present in another; the way I understand it, this ought to never run:

  if (images[event.queryStringParameters.img]) {
    return {
      statusCode: 200,
      body: images[event.queryStringParameters.img],
    }
  }

because images doesn’t carry over a value on subsequent instances of your Function.


:bulb::boom::exclamation:- While looking at your code further in my editor, I wonder if the issue is that you’re trying to pass binary into a web response. Wish I would’ve seen that sooner… it’s like the same reason we always JSON.stringify() JavaScript objects before sending them back in a response - fundamentally, everything that happens over an HTTP request is just a string. We see that in image uploads and downloads too - the browser just handles a lot of it automatically - but everything is passed around as base64 encoded strings.

I poked around the web for sending Node web responses that are buffers. I saw some folks using buffer.toString('base64') as one mechanism but I think you’re in the right direction with headers too. You are specifically sending back jpg’s, so your MIME type would probably need to be image/jpeg, but I think there’s more to it than just setting a MIME / “Content-type:” header. I don’t think I can full on deep dive with you here, but from a quick skim, I think you’re not far off and this article will give you (and me, whenever I get to reading it!) some foundational assistance - “Uploading and Downloading Files: Buffering in Node.js”. I also saw “Serving Remote PDF Files with Node.js and Express” and it looked pretty close to your goal as well.

Haha. Sometimes it dawns on me that once we take one step beyond HTML, CSS, and JS, we start getting into the nitty gritty of browsers and their long history of paradigms and “they just work that way”'s – image chunk fetching is absolutely one of them. My strongest recommendation would be to align yourself with those “ways” so that the browser just thinks it’s requesting a normal image (which, to your credit, I think you probably have right on the front end) that way you can leverage all the automatic chunking and what-not.

Please let me know how all of this goes. I’m really curious for your learninz on it :nerd_face:


Jon

Yeah… I think I literally just came to the same realisation as you did about the response type being binary. I started to mess around with base64 encoding it, and then jumped through a load of hoops to asynchronously get the text and populate the images in a separate react component. Once I figured out what was actually going on, I finally figured out that I could base64 it on the lambda end and give it the Content-type header, and then use the exact same front-end code I had in the first place (using a normal image and nothing more).

I feel kind of silly having not realised that I had to return a string, seeing as how I’ve jumped through that hoop so many times before. I’m aware its something on Netlify’s list of things to look at on the platform side, but I have a solution that works now, so that’s the main thing!

Thanks for your help. I really appreciate it.

No problem. Happy to help :slight_smile:

Not sure what you mean here though! ^