Getting the collection media_folder and public_folder to work

Netlify CMS
,
#1

I have been struggling to figure out how to get media_folder and public_folder working as a property of collections for days now. The docs only give a brief example and it’s still in the beta features section.

I don’t know if…
-This was a scrapped beta feature
-My cache is impeding (I imagine not since it’s cleared during builds)
-Gatsby is messing with it
-My local server configuration is broken (maybe something would work on deployment?)
-It’s only for folder collections? Or can you use it for file collections too?
-You can use it on fields (which I’d really like to do so I can have competition logos and winner pictures), but would open questions about how nesting works

I’ve tried all sorts of things, and combinations of them.
-Leaving it as a blank string and moving the images into the folder collection’s file with the markdown
-Clearing the cache explicitly
-Moving the images into a folder in static/img
-Using the {{public_folder}} and {{media_folder}} variables
-Using relative paths

I really don’t want to resort to putting all my assets into the root level of static/img. There’s dozens of them. It would get messy. There are images that are only relevant to particularly collections.

I don’t seem to be the only one struggling with this, as there’s still an open git issue for improving the docs on it.

#2

@fool I’m still struggling with this issue. I can’t seem to find anything else about this feature.

#3

Hi @theredwillow and sorry for the late reply.
Can you describe what is it that is not working? e.g the expected behavior and the actual result? Sharing a repo with the issue will be very helpful.

Have you tried using the configuration from the Gatsby guide:

#4

Thank you for replying, @erez. This issue has been my biggest blocker and I greatly appreciate your help. My repo can be found here.

Notice the configuration guide has their media_folder set to “static/img” at the parent level. I began by dumping all my images into that folder. It worked both on the site and within the CMS admin! :smiley: However, I quickly discovered that I had way too many images to keep them organized in one mere folder. Furthermore, everything I was using them for could be compartmentalized by how I was going to use it in the editor. This is the perfect application for individual media_folder’s, which are mentioned within the beta features in the docs.

The aforementioned configuration guide hints that media_collections can be tailored for each collection, but the supplied code uses an empty string (which defaults to the main one). So now I need to figure out how to configure these without its help.

The beta features page seems to suggest that these paths are absolute, so I tried to simply make one of my collection’s media_folder “static/img/competitions”. The idea being that anything related to competitions would be uploaded into that folder. However, when I visit the admin page, the preview starts to often bug out and the uploads appear empty when trying to select one, even after having added them in myself manually.

There is also mention of slug templates like {{media_folder}} that you can use to achieve the effect of relative paths. I also tried this, by setting one of my collections to something like “{{media_folder}}/competitions”. I came across the same errors.

It’s worth noting that Gatsby seems to always be able to find the images when I manually code them into my markdown (though there seems to be a nasty grab-n-dump quirk where folder location isn’t used in identification, so you have to treat each filename as a unique identifier).

The problems I’m having rest solely in the realm of the Netlify CMS admin editor.

Now assuming that I can even eventually get collection-based media_folders to work. I’d like to point out that my project’s structure actually lends itself better to FIELD-based media_folders. If this is a feature, I’d prefer to troubleshoot to get that working, but I don’t seem to see that in the docs. I’ll likely put in a feature request for it (would I do that through the Netlify github repository or the community?).

For example, if you follow along with my config.yml, you’ll see that competitions.image (the logos for various cook-off events) would benefit from pulling from a different folder than say competitions.years.winners.firstPlace (the pictures of previous years’ events’ first place winners).

#5

Using an empty string for media_folder doesn’t default to the main one (can you point me to the place in the docs that suggests that?) - media_folder: '' at the collection level means media files will be saved in the same directory as the entry.
The resulting file structure of the configuration in the guide is shown a bit after the config:

image
image1276×738 43.4 KB

Top level folders are absolute, collection level are relative. I believe this is described here.
Can you point me to section that suggests otherwise?

Best to submit feature request to the repository. You can have a look at this issue for more context.

#6

Thank you for your reply. As you’ve pointed out, I apparently glossed over some copy in the docs that I should have taken more time reading (like that the empty string defaults to the same directory and that there are relative paths in the collections’ configuration). That is a bad habit that I get into as a trial-and-error, hands-on learner. I need to think more like a bookworm before diving into new tech stacks. My apologies.

Let me lay out my project structure to make examining these issues a lil bit easier.

(root) > src > pages > cook-offs.md
(root) > static > img > competitions > burger.jpg

Using relative paths, I tried setting cook-offs’ media_folder to ../../../static/img/competitions and within the cook-offs.md file, there is image: /img/competitions/burger.jpg.

This allows the image to show up within the editor (notice it in the blacked-out background below), but not within the asset manager.

burgerworks
burgerworks1348×690 60.3 KB

This is the same error that happens when I try {{media_folder}}/competitions.

I also tried setting the media_folder to "" and converting cook-offs.md to a cooks-offs folder complete with index.md and a relocated burger.jpg file. The page worked within the admin page, but the same errors with the images were happening. Furthermore, the aforementioned configuration guide didn’t give any examples of media in the markdown, so I don’t know how the path works for it (since what was kinda working before was inside the static folder, not the src folder).

#7

I don’t know why I didn’t think to do this before, but I decided to just try publishing content to see what Netlify decides are the rules to its data structure.

As mentioned before, setting the global level media_folder to static/img, resulting in images being saved (root) > static > img.

However, when I went into one of my fields and added media_folder: '{{media_folder}}/competitions', it actually saved it to src/pages/static/img/competitions/thisisatest.jpg (this is a relative path to the markdown).

So the collection and field media_folder paths are ALWAYS relative, even if your global one is not. The {{media_folder}} template tag doesn’t do anything but copy the string, which breaks paths because they start from different places.

#8

Hi @theredwillow and sorry for the late follow up.

To force absolute paths you can prefix the configuration with /, for example:
media_folder: '/{{media_folder}}/competitions' at the collection/file/field will resolve to static/img/competitions (assuming the top level configuration is media_folder: static/img.

Please pay attention that the configuration needs to be either on the collection level, the file level or at the image field level, for example:


will not work as it is set on the list widget.

This was recently documented here.

Also hidden widgets are not supported in file collections so this doesn’t work:

#9

@erez Thank you for your follow-up. It makes me happy to think that other programmers facing these same misunderstandings might find this community resource!

I’d like to use an “uncle/aunt field” in my path. Specifically, the years list has the string field of “year” and I’d like the pictures to be stored in subfolders according to which list item year field value they have. I assume there’s no direct template tag to “parent’s field” (i.e. “{{…/fields.year}}”) and I’d have to use a cascading media_folder set-up. Is that right? Is that even a functionality, or does the media_folder template tag always fill with the root level one?

#10

Hi @theredwillow, template fields are evaluated per entry using dot notation from the entry root so you need to traverse down and not up, for example fields.top-level-object.child.another-object.year.

media_folder templates are cascading from root level to collection/file to the specific field.

fields. is used to access entry’s data while {{media_folder}} is used to access configuration (to avoid duplication)

#11

@erez When I set {{fields.year}} as the summary of years (its parent list), it is able to populate. That is incongruent with the absolute-path-style traversablity you described for the fields object. Do both fields-object-paths work?

will not work as it is set on the list widget.

This knowledge is complicating my issue quite a bit. I have a list, years, that contains objects. Each one of those objects has a year string that is a sibling to winners object. I can’t provide that year string to winner's descendants.

  1. Cascading: Impossible. I can’t grab the {{fields.year}} template like I did with summary, because media_folder's are impossible on lists.
  2. Traversing: Impossible. I can’t go down the list to get to my user input on year because the list is an array and it won’t know which index to grab.
              - name: "years"
                label: "Years"
                widget: "list"
                summary: "{{fields.year}}"
                fields:
                  - label: "Year"
                    name: "year"
                    widget: "string" # I'M THE SOUGHT-AFTER USER-INPUT
                  - name: "Sponsor"
                    label: "sponsor"
                    widget: "object"
                    optional: true
                    summary: "Sponsored by {{fields.sponsorName}}"
                    fields:
                      - {label: "Name", name: "sponsorName", widget: "string"}
                      - {label: "URL", name: "sponsorUrl", widget: "string"}
                  - name: "Winners"
                    label: "winners"
                    widget: "object"
                    # media_folder: "/{{media_folder}}/winners/{{fields.competitions.years[WHICH INDEX???].year}}"
                    fields:
                      - name: "First Place"
                        label: "firstPlace"
                        widget: "object"
                        summary: "1st: {{fields.winnersName}}"
                        fields:
                          - label: "Winner's Name"
                            name: "winnersName"
                            widget: "string"
                          - label: "Winner's Picture"
                            name: "winnersPicture"
                            widget: "image"
                            # media_folder: "/{{media_folder}}/winners/{{sought-after year user-input}}"
                      - name: "Second Place"
                        label: "secondPlace"
                        widget: "object"
                        summary: "2nd: {{fields.winnersName}}"
                        fields:
                          - label: "Winner's Name"
                            name: "winnersName"
                            widget: "string"
                          - label: "Winner's Picture"
                            name: "winnersPicture"
                            # media_folder: "/{{media_folder}}/winners/{{sought-after year user-input}}"
                            widget: "image"
                      - name: "Third Place"
                        label: "thirdPlace"
                        widget: "object"
                        summary: "3rd: {{fields.winnersName}}"
                        fields:
                          - label: "Winner's Name"
                            name: "winnersName"
                            widget: "string"
                          - label: "Winner's Picture"
                            name: "winnersPicture"
                            # media_folder: "/{{media_folder}}/winners/{{sought-after year user-input}}"
                            widget: "image"

Am I simply coming up to a limitation in the engineering of Netlify at this time? Will I simply need to devise a new folder structure for my winners’ pictures?

#12

I’m not sure what configurations the people in that github issue have, but Gatsby won’t let me continue without it. It’s the way it’s done in the starter boilerplate too.

#13

Hi @theredwillow, I think the confusing part is that we use the same mechanism (string templates) to access different types of data - configuration items (e.g. {{media_folder}}) and entry data (e.g. {{fields.year}}).

List summary works a bit different as it operates at a specific list item level, so {{fields.year}} traverses on a single list item and not the entire entry.

I also noticed that you use optional: true. I believe we only support required: false.

Another thing is that media_folder at the field level is only supported for image widgets, so setting it on an object (or any other widget) doesn’t do anything.

As for the problem at hand, we only supported specific indices when traversing, so I suggested opening a feature request to have a “special” kind of index to support that (or to suggest a syntax to traverse up instead of down).

I can’t think of a possible workaround other than restructuring the data. For example instead of a file collection with a list, a folder collection with an entry per winner.

#14

I decided to do a major architectural overhaul on a new branch until I’m ready to merge it over. I made a handful of lists on the cook-offs page and they hold relation widgets. The relation widgets point to src/cook-offs/ which hold contest folders. Each contest folder holds the markdown file, the contest’s logo image, and a folder of winners’ images.

Gatsby didn’t seem to be registering them as nodes in GraphQL, so I added a new gatsby-source-filesystem configuration for “cook-offs” in gatsby-config.js. To be honest, I am still learning the role of technologies such as these (gatsby-transformer-remark, gatsby-remark-relative-images, etc…). I think something is wrong with my commit.

The images in the cook-offs subfolders’ markdown are being imported as Strings, not images. This is evidenced by the classic error Field "image" must not have a selection since type "String" has no subfields upon running gatsby develop. There is also a warning about ENOENT: no such file or directory, open '/home/jared/Scripts/chili-w-i-chilly/.cache/json/_cook-offs_.json' which I do not understand at all. That file does not exist.

I found an article that describes some steps to fix images in Gatsby and Netlify, but I have already done all the steps mentioned. I’m not sure what to even Google to solve this problem myself.

#15

Hi @theredwillow, good progress with the branch.
To clarify:

  • media_folder tells the CMS where to put media files in the repo
  • public_folder tells the CMS what value to use in the markdown. This is later used by Gatsby to read the media file from the file system.

On your branch you’re saving media files in the same folder as entries, but pointing them to the wrong path.
For example https://github.com/theredwillow/chili-when-its-chilly/blob/11692e51acd598b5c950f469628cb1cf928624ce/src/cook-offs/burger-competition/winners/2017-burger-1st.jpg which is under src/cook-offs/burger-competition/winners but the markdown value points to /img/winners/2017-burger-1st.jpg:
https://raw.githubusercontent.com/theredwillow/chili-when-its-chilly/relation-widget/src/cook-offs/burger-competition/index.md.

To correct this, you simply need to remove the public_folder setting.

See PR: https://github.com/theredwillow/chili-when-its-chilly/pull/1

1 Like