This is more feedback than support request, but here it is.
I’ve setup NLM for the first time on one of my sites recently. I followed everything, read the requirements and limitations, setup, all of it.
After a while, here I was : time to push my first image to Large Media via LFS !
So I add an image, commit, push, it works. Netlify deploys it with success
Then I go to [id].netlify.app/images/.png and i get a 404. I check everything : the pointer file is on github, the url should be good. I clone the repo, git lfs pull, the image is here. But on the netlify dashboard’s Large media tab, it’s not. I wait a bit, maybe the servers are doing some magic on initialization and first time takes some time. I receive the “you started using NLM” email. it should work now!
It doesn’t.
Why doesn’t it work? Because I did not put the image inside the public folder, which is the only one deployed, per my deploy settings. I looked inside the docs, and nowhere it is said “warning: your files will only be available if you put them in your deploy folder, usually /public”. That was a frustrating noob issue I had, it could have easily been avoided with the right documentation.
I’m not sure which page should give us this warning. Either the “Usage and billing” page (but it’s more about billing than usage), or the “Transform images” page.
Hi, @Apollinaire. We do cover this in our documentation. All files deployed for any Netlify site are required to be in the publish directory when using the continuous deployment method:
Publish directory : Directory (relative to the root of your repo) that contains the deploy-ready HTML files and assets generated by the build.
If a file isn’t in the publish directory, it isn’t part of the deployed site.
We don’t currently highlight this requirement in the Large Media documentation because it isn’t a requirement specific to Large Media. This is always a requirement for all files, regardless of whether Large Media is used or not.
In other words, pointing out the requirement only in the Large Media documentation currently is not done because this requirement doesn’t have anything to do with Large Media itself. This was always a requirement even before Large Media is used and adding Large Media to a site doesn’t change this.
It might help us to understand where the misunderstanding is coming from. For example, answers to these questions below might help us improve the documentation:
Were you already aware of the requirement for all files to be in the publish directory when Large Media is not being used?
If “no” to the first question, would pointing this requirement out more clearly in the continuous deployment documentation be a helpful place to better highlight this?
If "yes to the first question, was there something written in the existing documentation which might imply this requirement would change when using Large Media?
I was aware of it, but it was just so seamless for me that I didn’t think about it. The SSG I use (ElderJS) builds to /public which is the default publish directory, so when I first deployed it just worked (the deploy). I added Large Media with LFS later on.
What confused me the most was that the git LFS service worked, so I thought “my images are on their server, it should be ok!”.
Another thing is that ElderJS builds to /public, but this folder is git-ignored. So it was literally impossible for me to upload something to Large Media, because you must commit your file to git in order to upload it.
I think it would be good to put a reminder inside the Large Media section. I’m not sure inside which page to do that exactly though. Setup page could be good because it’s a lot about git lfs, so you could say “While Netlify will cover all of your repo with LFS, only the files committed to your publish folder will be available to deploy & transformations”. To a further extend, you could, at the end of the page, tell the user to upload a file to their publish folder and check that it’s accessible online. That would make a good transition for the Transform Images page.
Last location could be Transform Images, with a Note or a Warning saying something like “The images will only be available on your site if they are deployed, so they should be committed to the publish folder”
Hi, @Apollinaire. I’ll ask our docs team to review this but I cannot promise we will include any of that verbiage above verbatim.
For example, I don’t think this is correct:
“The images will only be available on your site if they are deployed, so they should be committed to the publish folder”
The first part is accurate:
The images will only be available on your site if they are deployed
However, the second part of the sentence is not accurate:
so they should be committed to the publish folder
In most cases, the publish directory is intended to be empty and files should never be committed there. Also, most SSGs delete the contents of the publish directory prior to building a site so putting files there for most SSGs would guarantee that they would never be published.
The file under the publish directory are typically added to .gitignore specifically to prevent any files there from being tracked. Some static site generators (SSGs) use a specific directory for files to be copied to the publish directory and that would be the correct place to put these in most cases. For example, the Elder.js documentation says to use the ./assets/ directory and it will automatically copy them to the distDir (the default or whatever you have defined that to be).
However, each SSG is different so naming a specific directory in the documentation means listing dozens of possible “correct answers”. I don’t know that the documentation would be improved by listing a specific directory because there will be so many options to list.
I hope this clarifies why the documentation keeps things to a high level and why we answer questions about specific SSGs here on the community site. There is rarely any useful “one size fits all” advice. The only universal advice is “it depends on your SSG”.
I will bring this before to our documentation team about making changes to the documentation and we’ll take your suggestions under consideration.
Alright, your last message sheds light on something I did not know or even expect.
If I commit an image to /assets, it will pass through the LFS and be available through git, ok. But I thought that copying it from /assets to /public during build would break the Image transform service, because the file is not at the same location. I did not try this, so can you confirm it would work?
To clarify, here is an experiment: imagine that I chose to only track through lfs the *.jpg files. Then, I commit both .jpg and .png to my /assets, and my build copies them to the public folder. Both types will be available to download, but only .jpg will be available to transform? Similarly, if I don’t migrate to LFS my previously committed assets, they won’t be transform-able ?
What blows my mind is that the Image transform system is able to “know” how to match a file in the publish folder to its equivalent in the LFS server. Is this based on the hash made by git-lfs?
If, while moving an image from /assets to /public I also rename it, will transforms still work?
All this to finally conclude that the core issue is that your system is so magical that it can work in ways that I did not think was possible, congratulations on that!
I’ll try by myself some of the experiments above tomorrow, thank you for the amazing clarifications already!
Hi, @Apollinaire. I wanted to explain more about this:
The way it works behind the scenes is the also the reason for the “no image processing at build time with Large Media” limitation.
In the build image, the original file is replaced with the Git LFS text pointer during the repository cloning. This why build time image processing isn’t possible. The image isn’t there during the build as it was replaced with the text pointer.
The pointer files do, however, unambiguously identify the original file. If they are moved, we still know which file it is - regardless of where it is moved to. After deployment, we know which file to serve. (In fact, one file might be copied to several locations for a single deployed site, if someone had a reason to do so.)
Note, this is a Large Media limitation only. If you use the Git host’s LFS service, the real file is downloaded when the repository is cloned. In this case, build time image processing is possible but browse time image transformation provided by Large Media is not.
That makes total sense; when I think i’m moving the image, i’m actually moving the pointer file which is why it’s possible for NLM to “know” that it’s a Large Media file.
Thanks a lot for the explanations!
@luke I would like to kindly +1 the request for more detailed docs on the details underlying the large media architecture.
While it is clear from the netlify docs that all assets need to be in the project deploy folder, it’s fairly challenging to make the leap to the fact that with large media your images will be text pointers on the netlify build server at build time. This means you need to move them into the deploy folder using a method that just moves their bytes and does not parse them as images.
I am still trying to work out how to do this using webpack (the thing that assembles my deploy folder) since by default webpack is expecting valid image bytestreams even when image processing options are disabled.
Thanks, good point! Luke has a rewrite of our large media docs on his roadmap so we’ll get to it as soon as we can, and I am sure he will include that difference between local builds and ones in our CI
.png and i get a 404. I check everything : the pointer file is on github, the url should be good. I clone the repo, git lfs pull, the image is here. But on the netlify dashboard’s Large media tab, it’s not. I wait a bit, maybe the servers are doing some magic on initialization and first time takes some time. I receive the “you started using NLM” email. 
it should work now!