[Support Guide] Troubleshooting your Netlify Large Media Configuration

Support Support Guides
,
52

Hi there! Is this the megathread for troubleshooting with Netlify Large Media?

I have LFS all set up, but I can’t access my file from the site. This is file I’m trying to access:

It returns a JSON 404 object {"code":404,"msg":"File not found"} as opposed to the usual 404 HTML page.

In my app’s Large Media tab, it’s showing this file (and three others — all of which don’t work).

I can also share my repository, which is open. You can see the three files here, checked in with Git LFS: vision2020/public/media/attract at main · BadIdeaFactory/vision2020 · GitHub

And this is the output when I run lm:info:

➜  vision2020 git:(master) netlify lm:info
  ✔ Checking Git version [2.24.0]
  ✔ Checking Git LFS version [2.8.0]
  ✔ Checking Git LFS filters
  ✔ Checking Netlify's Git Credentials version [0.1.8]

Everything seems to be working! But the fact that I still can’t access these files must mean there’s something I’m not understanding about the way Large Media works.

If it makes any difference, I’m using Next.js as my build process. I do not need access to the files during the build. I only need them to be hosted, so that my site’s front end can access those files.

Thanks!

53

Hi, @louh, and welcome to our community site.

It is definitely okay posting here or you can create a new topic with your Netlify Large Media (NLM) question (in the future or if other people are reading this and want to ask NLM questions too). Both work great so do whichever you prefer.

As you stated, I also see the files as expected in the Large Media configuration page in the Netlify UI, but the URLs are not working when I test.

Would you please check that there is a file named .lfsconfig in the base of the git repo for this site?

It should contain something similar to this:

[lfs]
	url = https://8eab2635-9055-47af-95b2-0e81748a8a36.netlify.com/.netlify/large-media

Note, your URL should be different and the netlify.com subdomain should be the same as the API ID found here:

(The API ID is not sensitive and can be publicly shared without concern. The API keys are sensitive so please never share those, but site API IDs are safe to share publicly.)

If the .lfsconfig file is there (and it probably is), next please confirm that you have the actual binary file in your local repo (not the Git LFS pointers which are three lines of text). If you have the actual files locally in the repo, would you please try running the following command to force their “re-upload” to Netlify:

git lfs push --all origin

Then trigger a new deploy of the site and let us know the MP4 URLs are working after this is done.

If it still isn’t working, please run one more command (below) but do not post the results here in public. Please DM/PM me (@luke) the output of the command. I’ve made sure you can direct message but if you cannot for some reason, let us know here. The command we need the output for is the one below:

GIT_TRACE=true GIT_CURL_VERBOSE=true git lfs push --all origin

That is the debugging version of the first command and it will tell us more about what is happening if the first command doesn’t fix the issue.

It does resolve the issue in many (not all) cases and, if there are additional questions about this, please add them anytime.

1 Like
54

Hi @luke! Thank you for the thorough response. This worked:

Would I need to re-run this every time I have new media, updated media, or just whenever things seem to broken?

55

Hi @luke I’ve run through your steps provided and still seem to being having issues seeing my files in the Netlify Large Media tab of my site dashboard.

I can confirm that I’m tracking files, I have my .lsfconfig file in the root directory of my project, I have the correct Netlify API ID in my .netlify directory, I’ve migrated and pushed all of my lfs tracked files using the provided commands in the documentation with no errors, but I still am not seeing them in my Netlify dashboard. That said, I am testing this on an auto generated Netlify site as I’ve had to fork my main repo for testing. EDIT: I just read the docs again and saw that this won’t work with a fork of the repo.

Is there a solution to migrate from not using Netlify Large Media without pushing to my master branch of the main repo? Since I wasn’t using LFS previously, It will break my website until I get changes in that remove gatsby-image from the entire site.

Let me know if I’ve missed anything here.

56

@louh, you probably won’t need to use this again.

I believe the issue in this case most commonly occurs when one Git LFS provider (like GitHub’s LFS implementation) is used for the repo and then later the LFS service is moved to Netlify. The repo thinks the files have been uploaded once already (because they were - just not to Netlify) so git only sends the “metadata” - not the actual image itself.

The command above forced the push of the actual image data and resolved the issue.

As new images added in the future won’t be marked in the repo as uploaded already (because they’re new) this shouldn’t happen again. If it does happen again with this repo, please do let us know.

1 Like
57

@bobeartow, are you wanting to remove the Netlify Large Media (NLM) add-on from one or both sites at Netlify?

If so, this is done by removing the .lfsconfig file, making any other required changes to the repo (for example, configuring it to use a different Git LFS service or no Git LFS service at all), and then pushing those changes to a branch configured to deploy at Netlify to see if the new deploy is successful.

The new deploy will show the LFS pointers instead of the actual file contents. Then we disable the NLM add-on from our side and ask you to trigger another deploy. This deploy should show the actual contents of the files now that the add-on is removed - in other words, it should be the working site with all files displaying correctly.

If this latest deploy has all LFS assets working, then the branch is merged into the production branch as well. The production branch can be any branch. In practice, it is most often “master” - but it doesn’t have to be.

Some people skip the branch testing and test these changes directly on the production branch.

Another option to stop using NLM for a site/repo is to delete the site at Netlify, make the changes in the repo, push those changes upstream, and then reconnect the repo to a new site. The new site won’t have NLM enabled but it also won’t have the deploy history of the previous site. This is more of a “brute force” method but it also can be done without assistance from our technical support team. This obviously also means potential downtime for any custom domains as they will need to be added to and configured to use the new Netlify site.

Please note it is very important to be certain you have copies of the actual files (not the pointers) locally before proceeding with either method. This is because the copies of those files at Netlify will not be able to be recovered once the site or NLM add-on is removed.

Would you like to proceed with one of these methods? If so, please let us know which site you want to make these changes for and which method you want to use for the removal of the NLM plugin. If there are any questions, we’re happy to answer.

58

It’s happening when you don’t have the git-credential-netlify installed. I don’t know why it happened but I had the same issue. You can find all the details to manually install it here :

1 Like
59

Hi have tried all the suggestions in this thread but still no joy. Continually getting the ‘batch response: Rate limit exceeded:’ error. git lfs ls-files does return a list of tracked files so it would appear my configuration is otherwise ok.

Is the ability to track nested directories now supported? Any help with the above greatly appreciated!

60

Hi, @brianm, and welcome to our Netlify community site.

Does the command netlify lm:info show the Netlify Large Media (NLM) add-on being configured for this repo locally?

If not, please have look at the NLM setup instructions here.

If the NLM is configured for this repo/site, would you please send us the URL for the deploy logs where the “Rate limit exceeded” message appears?

Regarding the ability to track nested directories, my understanding is that NLM has always supported this. If you add a pattern with git lfs, for example with git lfs track '*.png', then PNGs will be tracked regardless of what directory or subdirectory they appear in.

Then again, maybe I’m misunderstanding what you meant by “nested directories”. If I am misunderstanding, would you please clarify this for me?

And, as always, if there are other questions please feel free to include them anytime.

61

Hi @luke yes that cmd shows NLM configured correctly and when I look at verbose output of the failing git push it again shows the rate limit exceeded error.

Re. nested directories, I thought I’d seen a post above mention that nested directories may not be supported and wanted to rule that out as a cause of error (i’ve since deleted those nested directories). My intention was to track files all files and subdirectories within a particular directory. The site is a migration from WP so was hoping for the time being to keep the WP uploads directory structure intact (wp-content/uploads/2020/10 etc. so had tried wp-content/uploads/* but not sure if that is sufficient.

Finally the url in question I think you are looking for re. logs is:
https://6edd00f8-be81-4516-95d2-32a0cca9a15c.netlify.com/.netlify/large-media/objects/batch

63

Hi, @brianm. We emailed you about this because we might need to share sensitive details which shouldn’t be shared publicly.

If you don’t receive the email though, please reply here to let us know,.

64

Hi @luke ended up going for the nuclear option and created a new site with content served from static directory instead. Going to blitz the old site and attempt large media again in a few days, if I run into same issues, will ping the verbose logs.

1 Like
65

sounds good @brianm. Let us know if you run into issues - i recommend starting a new thread so that it doesn’t get lost :wink:

66

Hi! Need some help troubleshooting.

I see images listed under the “Large Media” tab:

Screenshot 2020-02-02 at 00.56.38
Screenshot 2020-02-02 at 00.56.381880×846 67.3 KB

But the images don’t appear on the site where I expect: https://18thcenturyblog.netlify.com/images/wolves.jpg (this is a 404 at the time of writing)

Do I have the wrong path? Do I need to wait longer?

I’m on a free plan, if that makes any difference.

EDIT: Ah, found a similar situation above and am currently trying the suggestions in [Support Guide] Troubleshooting your Netlify Large Media Configuration - #53 by luke …

EDIT 2: Yup, git lfs push --all origin solved it.

67

Hi, @henrik, welcome to our Netlify community site.

Thanks for sharing the issue and the solution which worked for you. Hopefully, others searching for similar issues will see this and benefit. Thanks again!

68

Hi Luke; thanks!

I wonder if the UI could be clarified here to make this less of a gotcha. Could the “Large Media” tab detect and indicate this situation, for example?

69

hi henrik, thanks for your feedback - we’ll make a note of it for the future!

70

Hi Luke, I’ve followed the instructions here and I’ve had no progress. There’s a video file in my site that plays in the background. Its not showing up when deployed. The command git lfs ls-files shows that the file is being tracked. However it doesn’t show up in the Large Media Tab. Could you help me troubleshoot?

Screenshot from 2020-02-16 16-56-38
Screenshot from 2020-02-16 16-56-381366×768 103 KB

71

Hi, @tyirenkyi, and welcome to our Netlify community site.

First, would you please confirm that your local Git repo contains the actual file and not a pointer file (which will be a three line text file and much smaller than a video file).

If it does, please run the following command to upload the file to Netlify Large Media (NLM):

git lfs push --all origin

Then trigger a new deploy of the site.

If this still doesn’t resolve this issue, please run the following command but do not post the results publicly:

GIT_TRACE=true GIT_CURL_VERBOSE=true git lfs push --all origin

Then, either let us know here that you are ready to send that to us or send that to us via private message (PM).

You can PM that to one of our support staff and I’ve confirmed that PMs are enabled for your community login. Note, that only one person can see the PM and this will likely mean a slower reply than posting the information publicly. Please feel free to reply to however you prefer though.

72

I followed your instructions

  1. I can confirm that my local repo contains the actual file and not a pointer file.
  2. I ran this command, git lfs push --all origin.

Uploading LFS objects: 100% (1/1), 42 MB | 0 B/s, done.
The above was printed out in the terminal.

  1. I ran this command
    GIT_TRACE=true GIT_CURL_VERBOSE=true git lfs push --all origin
    I am ready to send the results to you guys.

Thank you for your help