[Support Guide] Troubleshooting your Netlify Large Media Configuration

Hi, @somratpro. If you are trying to do something with another service it probably doesn’t have access to the Large Media instance (the Git LFS instance at Netlify).

You might need to disable Large Media for this site and use Git LFS without Large Media. Have you tested that?

If you want to test uninstalling Large Media for this site, please read this support guide:

If you have read that, please confirm you have backups of all files and let us know if we should remove the Large Media add-on from this site.

hey Luke,

I solved this problem by using GitHub. Gitlab is very complicated with those types of issues.
Thanks a lot.

1 Like

I’ve tried migrating over from using BitBucket to Netlify for LFS and am running into the same kind of issues seen above. To be safe I decided to try migrating in a separate branch and now builds on master are failing.

git checkout netlify-large-media 
git lfs fetch --all # trying to pre-fetch all the files before configuring to use NLM
netlify lm:setup
git lfs migrate import --everything --include="*.jpg,*.png,*.encrypted" # *.encrypted is needed for BitBucket Pipelines
git push --force-with-lease
git lfs push --all origin
git lfs push --all origin netlify-large-media
git lfs push --all origin master

Trying to avoid going the nuclear option and nuking the site.

Hi, @mbeigel. Large Media doesn’t have a concept for “enabled for only this branch”. Large Media is enabled at the site level and then applies to all branches being built at Netlify. You cannot enable it for some branches and not for others.

If you want to test Large Media, I suggest creating a new repo for testing purposes because it won’t be possible to test only for a single branch.

Do you want to remove the Large Media add-on from the existing site for now? If so, please read the support guide here first:

and then let us know then you are ready.

If there are other questions about the current configuration, please let us know.

Hello, I’m unable to run netlify lm:install successfully (windows10)
My problem closely resembles that of jjazzboss

npm i netlify-cli -g
netlify plugins:install netlify-lm-plugin
netlify lm:info
√ Checking Git version [2.29.2.windows.1]
√ Checking Git LFS version [2.12.0]
√ Checking Git LFS filters
× Checking Netlify's Git Credentials version
→ Check that Netlify's Git Credential helper is installed and updated to the latest version

I then download the credential helper manually with powershell. (GitHub - netlify/netlify-credential-helper: Git credential helper to use Netlify's API as authentication backend)

netlify lm:info
√ Checking Git version [2.29.2.windows.1]
√ Checking Git LFS version [2.12.0]
√ Checking Git LFS filters
√ Checking Netlify's Git Credentials version [0.1.9]

netlify lm:install
√ Checking Git version [2.29.2.windows.1]
√ Checking Git LFS version [2.12.0]
√ Checking Git LFS filters
× Installing Netlify's Git Credential Helper for Windows
→ spawn EPERM
Configuring Git to use Netlify's Git Credential Helper
Error: spawn EPERM
at ChildProcess.spawn (internal/child_process.js:406:11)
at Object.spawn (child_process.js:548:9)
at module.exports (C:\Users\gefyo\AppData\Local\netlify-cli\node_modules\netlify-lm-plugin\node_modules\execa\index.js:139:26)
at Task.installWithPowershell [as task] (C:\Users\gefyo\AppData\Local\netlify-cli\node_modules\netlify-lm-plugin\lib\install.js:85:12)
at C:\Users\gefyo\AppData\Local\netlify-cli\node_modules\netlify-lm-plugin\node_modules\listr\lib\task.js:167:30
at processTicksAndRejections (internal/process/task_queues.js:97:5) {
errno: -4048,
code: 'EPERM',
syscall: 'spawn',
context: [Object: null prototype] {}
}

I never seem to see a git-config file in my /users/gefyo/.netlify folder
I do see the credential-helper in .netlify/helper/bin/ after manually installing it

$env:path C:\Python38\Scripts\;C:\Python38\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\Amazon\AWSCLIV2\;C:\Program Files\nodejs\;C:\ProgramData\chocolatey\bin;C:\Perl\c\bin;C:\Perl\perl\site\bin;C:\Perl\perl\bin;C:\Program Files\Git\cmd;C:\Users\gefyo\scoop\shims;C:\Users\gefyo\AppData\Local\Microsoft\WindowsApps;C:\Users\gefyo\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\gefyo\AppData\Roaming\npm;C:\Program Files\NASM;C:\Users\gefyo\.netlify\helper\bin

I’ve tried doing quite a few uninstall variations and logouts, login, link, etc.
I will get this error as well after a netlify-cli reinstall

netlify lm:install
√ Checking Git version [2.29.2.windows.1]
√ Checking Git LFS version [2.12.0]
√ Checking Git LFS filters
× Installing Netlify's Git Credential Helper for Windows
→ Command failed: powershell -ExecutionPolicy unrestricted -File C:\Users\gefyo\AppData\Local\Temp\netlify-cHg23F\in...
Configuring Git to use Netlify's Git Credential Helper
Error: Command failed: powershell -ExecutionPolicy unrestricted -File C:\Users\gefyo\AppData\Local\Temp\netlify-cHg23F\install.ps1 -windowstyle hidden

netlify lm:setup
√ Provisioning Netlify Large Media
√ Configuring Git LFS for this site

I proceed to git lfs track, etc, but upon git push I get the Windows Git Credential Manager prompt

Hi, @gty. These look like permission issues from the OS when running the Git Credential Helper install commands.

Have you tried running this in a Powershell instance with Administrator level permissions?

Edit: I don’t think running that as admin is a good long-term solution but it would rule out OS permissions if it still doesn’t work. However, if it still doesn’t work as admin I would expect a different error at least so please let us know if that changes anything or if there are other questions.

Thanks for the help @luke I had tried running in administrator mode and received the same error. However I just turned off my anti virus and it installed!

Thank you for sharing this solution! This kind of information is extremely helpful for other people searching this community site.

Our support team appreciates you taking the time to follow-up to share the solution you discovered and I believe those “future web searchers” will thank you as well. :+1:

When I upload an image via the web CMS at my-site-name.netlify.app/admin, the image is not picked up by the Large Media service. For example:

I have just uploaded a file called everything-happens-for-a-reason.jpg in the image > choose an image > upload new workflow.

After the build finishes, I go into my local repo folder, run git pull and see this:

image

Sure enough, the entire image file is there, not the pointer. What’s more, git sees it as being modified.

I check existing large media at app.netlify.com/sites/my-site-name/large-media and it is not listed.

Locally, I commit the change and git push. Once the build has run, I return to the Large Media list and the file is there!

So the LFS is working with local git actions, but not from the web-based CMS.

Running netlify lm:info returns:

√ Checking Git version [2.20.1.windows.1]
√ Checking Git LFS version [2.6.1]
√ Checking Git LFS filters
√ Checking Netlify’s Git Credentials version [0.1.9]

Here is my .gitattributes :

public/images/** filter=lfs diff=lfs merge=lfs -text

And my .lfsconfig is correct as well, ending in .netlify.app/.netlify/large-media.

One more thing: Once I have gone through all the steps below ( git push etc) so that the image files DO show up in the large media tab, their thumbnails break in the image picker in the CMS.

That’s all I know. Can you please advise? Thanks!

Hi, @thischrisblack. Which CMS are you using and what site is this for?

If you are using Netlify CMS, version 2.6.0 or newer should support Large Media and there is more information here:

https://www.netlifycms.org/docs/netlify-large-media/

If you are using a different CMS, please let us know. Regarding the site, you can send us the API ID for the site if you don’t want to post the subdomain.

We look forward to your response and please let us know if there are other questions for us.

1 Like

Hi @luke! The site is tender-mestorf-e583fe using nextjs. I’ve got this in the admin/index:

<script src="https://unpkg.com/netlify-cms@2.9.7/dist/netlify-cms.js"></script>

If I add an image via the CMS (with “add an image” > “choose image” > “upload new” in a blog post, for example), the whole image file goes in the repo and the image does not appear in the "Large Media: tab. But if I manually place an image in my public/image folder locally and then commit and push it, it appears as a pointer in Github and also appears properly in the Large Media tab.

Thanks for the help!

EDIT: I should also mention that after performing the local git push so that the image appears in the large media tab, its thumbnail is then broken in the “choose image” modal in the CMS, even though the image shows up properly in the blog post itself. It’s all kind of weird.

@luke RESOLVED!

Digging through this thread I thought I’d better check my import and sure enough I was:

<script src="https://unpkg.com/netlify-cms@2.9.7/dist/netlify-cms.js"></script>

I updated to:

<script src="https://unpkg.com/netlify-cms@^2.0.0/dist/netlify-cms.js"></script>

And everything works now.

PLEASE NOTE: I copied that admin/index.html code straight from the Netlify CMS docs for NextJS. I’ll submit an update to those docs shortly. (Update: See docs PR.)

Thanks!

1 Like

Hi,
I just setup my gatsby project to use Netlify lm. But I don’t see the files under the Netlify Large Media tab. My build also fails but I think this is due to the gatsby-image package.

  • netlify lm:info
  ✔ Checking Git version [Git-128)]
  ✔ Checking Git LFS version [2.10.0]
  ✔ Checking Git LFS filters
  ✔ Checking Netlify's Git Credentials version [0.1.9]
  • my .lsfconfig
    https://<hash>.netlify.app/.netlify/large-media

  • my .gitattributes is set up correctly

  • I see images in my local repo but on github I see the pointers

  • I’ve executed git lfs push --all origin

  • asset optimization is disabled

Any idea what’s going on?

UPDATE:
It turns out it was the build that was causing the issue. Here’s how I managed to fix it.

  • Removed all references fo gatsby-image Img component and covert it to basic <img /> jsx
  • npm uninstall gatsby-transformer-sharp gatsby-plugin-sharp
  • refactor the code accordingly.

I’ve also managed to use netlify’s image transformation services since I have a single component handling all the images.

Hi, @samarara. (I like the forum username by the way.) I believe that when using of these two packages that they are doing work with the raw image data during the site build:

  • gatsby-transformer-sharp
  • gatsby-plugin-sharp

It isn’t possible to use the raw image data during the site build if Large Media is being used. The solution above appears to have been to stop doing build time transformations and utilizing browse time transformations instead.

Note, there is another solution which would allow you to use the two packages above. That solution is to use Git LFS but not use Large Media.

Large Media is a Git LFS service at Netlify which allows browse time image transformations but using Large Media when Git LFS is used is optional. Using Large Media is not a requirement when using Git LFS.

You can also use Git LFS but continue using the Git service’s Git LFS service (and not use Large Media at all). By default, the Git service (GitHub, GitLab, Bitbucket) provides the Git LFS service for the repo.

If this is done (Git LFS without Large Media), the raw image data is available during the build and build time image transformations do work (but the browse time transformation will not as that is provided only by Large Media).

Thanks for sharing your solution in the follow-up. If there are any other questions about this, please let us know and we’ll be happy to answer.

Thanks so much! Had the same problem, spent hours trying to figure out what was wrong with my git configuration. Of course I should have realized this, but still: would be good to add an explicit warning to the settings page where one can switch on Asset Optimization (or better even: don’t allow users to turn it on when using Large Media).

Hi, @elmarj. There is an open feature request for exactly this (don’t allow one to be enabled if the other is in use).

If or when this change happens we will post an update here.

1 Like

Hi dear support & thanks for the guide @luke

I have set up a site/repo for netlify large media today, in order to being able to use transformations and publish the website.

I’ve managed to get it all running locally, in CLI, and uploading the tracked images to LSF seems to be working (also checked all things you OP mentioned and it’s all green checks).

but, since everything has been set up, my website builds are failing on push.

I have also added the env variable GIT_LFS_ENABLED to true, which seems to work.

I’m attaching the build error log, which seems very generic to me and I have no idea how to go on from here :slight_smile: any advice on debugging this further?

Thanks!
N.

6:01:53 PM: Build ready to start

6:01:56 PM: build-image version: d84c79427e8f83c1ba17bcdd7b3fe38059376b68
6:01:56 PM: build-image tag: v3.6.1
6:01:56 PM: buildbot version: 44655717ddf0e7bd7f856f5b1154254de54b1d80
6:01:56 PM: Fetching cached dependencies
6:01:56 PM: Starting to download cache of 1.2GB
6:02:01 PM: Finished downloading cache in 5.544915992s
6:02:01 PM: Starting to extract cache
6:02:20 PM: Finished extracting cache in 18.693005743s
6:02:20 PM: Finished fetching cache in 24.552205934s
6:02:20 PM: Starting to prepare the repo for build
6:02:20 PM: Git LFS enabled
6:02:21 PM: Preparing Git Reference refs/heads/master
6:02:30 PM: Starting build script
6:02:30 PM: Installing dependencies
6:02:30 PM: Python version set to 2.7
6:02:30 PM: Started restoring cached node version
6:02:34 PM: Finished restoring cached node version
6:02:34 PM: v12.18.0 is already installed.
6:02:35 PM: Now using node v12.18.0 (npm v6.14.4)
6:02:35 PM: Started restoring cached build plugins
6:02:35 PM: Finished restoring cached build plugins
6:02:36 PM: Attempting ruby version 2.7.1, read from environment
6:02:37 PM: Using ruby version 2.7.1
6:02:38 PM: Using PHP version 5.6
6:02:38 PM: Started restoring cached node modules
6:02:38 PM: Finished restoring cached node modules
6:02:38 PM: Started restoring cached yarn cache
6:02:38 PM: Finished restoring cached yarn cache
6:02:39 PM: Installing NPM modules using Yarn version 1.22.4
6:02:39 PM: yarn install v1.22.4
6:02:39 PM: error An unexpected error occurred: “/opt/build/repo/package.json: Unexpected token v in JSON at position 0”.
6:02:39 PM: info If you think this is a bug, please open a bug report with the information provided in “/opt/build/repo/yarn-error.log”.
6:02:39 PM: info Visit yarn install | Yarn for documentation about this command.
6:02:39 PM: Error during Yarn install
6:02:39 PM: Build was terminated: Build script returned non-zero exit code: 1
6:02:39 PM: Failing build: Failed to build site
6:02:39 PM: Failed during stage ‘building site’: Build script returned non-zero exit code: 1
6:02:39 PM: Finished processing build request in 43.447594476s

update! my bad, I accidentally added ALL files to large media, instead of just the images… this broke the deployment of course.

now all images seem to be stored with git ls, at least in the git repo. but the large media section on netlify still isn’t activated (Netlify App) somehow and the image transformations are not working therefore.

Hi,
I just initialised my repo to use lm. But when I’m trying to push, I get the error Aborting Netlify credential helper execution and TLS handshake timeout. It then asks for username and password (I tried to fill out, but with no success).

I tried to re-initialise everything, but now when I run netlify lm:setup it throws TextHTTPError: Unauthorized.

What can I do to make it work?

1 Like