GridFs is NOT working on production/fails to read files from storage

Support
,
1

Hello there.,
I’m new here and this is going to be my very first topic opening up not only here but in all tech community forums.

So, I have a Express app which works as an API service (provides and stores data to MongoDB atlas instance). I’ve deployed it on Netlify using Netlify Dev. Everything works as expected except the route that handles filename fetching from GridFs storage. File itself is an image and after getting it’s name from gfs storage it should be displayed on client side. On heroku, on localhost, also after running netlify dev it works perfectly fine but after running netlify deploy it fails. Behaves weirdly that I don’t understand. When I’m hitting the endpoint to read the file I’m getting three different kind of messages although I do change nothing.

  1. TypeError: Cannot read properties of undefined (reading ‘find’) (happens sometimes)
  2. ERROR: Function crashed (which happens rarely)
  3. Downloads binary file with the name of filename which I provide as URL parameter. (happens often)

Here is the code that handles that route.

route.get("/image/:filename", (req, res) => {
        gfs.find({filename: req.params.filename}).toArray((err, files) => {
            if (!files[0] || files.length === 0) {
                return res.status(200).json({
                    success: false,
                    message: "No files available"
                })

            }

            if (files[0].contentType === 'image/jpeg' || files[0].contentType === 'image/png' || files[0].contentType === 'image/jpg') {
                gfs.openDownloadStreamByName(req.params.filename).pipe(res)
            } else {
                res.status(404).json({err: "No Image Found!!!"})
            }
        })
    })

Link to app

this is the endpoint to reach the route handler
/functions/index/api/upload/image/cc3fa9f207d6a2e6c5be0

I’m not sure if I provided enough information or even explained it in a comprehensive way. Hope, I’ll get some help.

2

Hey @salyut,

Super sorry for the delay. This thread got out of our tracking system, because of which we missed it. It has been added back, so it should not happen going forward.

Talking about your issue, do you think you can share the repo? I’m not particularly experienced with GridFS, but the above piece of code looks like something that should work. The error that you mention:

TypeError: Cannot read properties of undefined (reading 'find')

Seems to be coming from: image.js:48:13. In the code you’ve shared, I can see .find() being used here:

Is that line 48?

3

Hey @hrishikesh! Happy to hear from you.

Here is the link to repo radish-garden-api

Yes. This Error is coming from image.js:48:13. Somehow, sometimes on production a value that find() gets called happens to be undefined. (It never happens on local).

4

I can assume that the event handler that you have added either never triggers or something goes wrong. The Mongoose documentation doesn’t seem to be developer friendly, at least not at the first glance. For example, I don’t know where the .once comes from:

https://mongoosejs.com/docs/search.html?q=.once

In any case, I wonder, why don’t you use the MongoDB driver itself: https://www.mongodb.com/docs/drivers/node/current/fundamentals/gridfs/

Seems much more modern and well-maintained.

5

These days I delved into concepts of GridsFS as much as I could and tried some of different ways to make it work. As you suggested I changed configuration to MongoDB driver itself but problem remains the same though I don’t get this error anymore.

TypeError: Cannot read properties of undefined (reading 'find')

Also never getting Netlify internal Error about function crushing as it happened before
but instead of rendering an image on the browser downloads an unknown type of file.

When I make request through postman it displays raw data. Reads the stream instead of downloading.
On the local host behaves as expected. Renders the image.

7

Hey @salyut,

I setup a basic GridFS test repo and it appears to work fine. Here’s the output:

https://dashing-sunshine-b46b48.netlify.app/.netlify/functions/list

and here’s the source:

8

Thanks @hrishikesh for your help and support. I totally changed my approach not using GridFS anymore.
Thanks also for sharing your code it works but I don’t see that it solves the problem I had initially. I had no issue with retrieving a document from the collection(db).

I also set up same approach I had before and getting the same result.
website URL
GitHub repo

My intention was storing images on db through GridFS and then reading (rendering) them on the browser without downloading the file itself.

bucket.openDownloadStream(_id) returns a readable stream (GridFSBucketReadStream) for streaming file data from GridFS. So after using pipe() method bucket.openDownloadStream(_id).pipe(res) image should get rendered on the browser directly or after passing “reader route” to src on img element <img src="http://<some url>/:fileId" />

Tested on Digital Ocean and it works as expected. I don’t know whether it is netlify related issue or I’m doing something not in a right way but my question on why has no answer yet.

9

My code was not meant to solve your use case, I was trying to test if .find() works and based on my testing, it does work.

So based on your previous code here: https://github.com/giga-mania/radish-garden-api/blob/production/deploy/netlify/functions/routes/image.js, I can only imagine that you were setting the value of gfs incorrectly.