Parse this article format (Pelican, Markdown)

I have a Pelican blog and I am attempting to use Netlify CMS with it. My article format looks like this:

title: My Article Title
slug: my-article-title
date: 2020-01-08
author: William

This is some *amazing* article content.

And my config.yml looks like this:

...

collections:
  - name: articles
    format: yaml-frontmatter
    frontmatter_delimiter: ''
    ...

Netlify CMS doesn’t parse any information from the existing files. Is there some combination of configuration options that will make this work? Or, do I have to have something besides an empty line to delimit my front matter?

Have a look at the (fairly old) Pelican starter at the Netlify Templates repo: https://github.com/netlify-templates/pelican-netlify-cms/. It uses format: markdown-python. It’s the first time that I come across that format option and I’m 99% sure it’s not documented, but it might just work.

I had not found that repo, so thanks for bringing it to my attention. Unfortunately, it looks like it is for https://github.com/netlify/netlify-cms-legacy. If I try format: markdown-python with the latest version of Netlify CMS I get:

Error loading the CMS configuration

Config Errors:

'collections[0].format' should be equal to one of the allowed values

Check your config.yml file.

Yeah, only noticed after I posted that comment that the repo uses v1. Would it be possible to add a frontmatter delimiter to your markdown files? I found a repo that does this (here), though I’m not sure how. I’ve never used pelican.

Thanks for you help. I probably won’t go back and update all my files to include delimiters right now, but I will always remember that as an option for the future.

For the benefit of anyone who finds this topic in the future, I just wrote a quick script to add delimiters to all my files:

import pathlib

path = pathlib.Path.cwd()

for item in path.iterdir():
    if item.suffix == '.md':
        new_lines = ['---\n']
        with item.open() as f:
            metadata_complete = False
            for line in f:
                if metadata_complete:
                    new_lines.append(line)
                else:
                    if line == '\n':
                        new_lines.append('---\n')
                        metadata_complete = True
                    else:
                        new_lines.append(line)
        with item.open('w') as f:
            f.writelines(new_lines)

This did not affect Pelican at all, and now Netlify CMS can parse the files, too.

2 Likes

thanks for sharing this, William! This will undeniably come up again.

I’m using Flask-FlatPages and Flask-Freeze and running into the same error. The delimiter script didn’t work in my case. Markdown works as expected as for the OP, but I can’t seem to set FLATPAGES_MARKDOWN_EXTENSIONS to the right extension to support what comes from Netlify CMS. Neither python-markdown-yaml-meta-data nor python-markdown-full-yaml-metadata work.

I can manually create my own Markdown files in the format Flask expects of course, but I had wanted to use Netlify CMS as a frontend rather than rolling my own headless CMS. Any tips?

Right now if I run a Netlify CMS Markdown file in FlatPages I get this error:

yaml.composer.ComposerError: expected a single document in the stream
in "<unicode string>", line 2, column 1:
title: Hello World
^
but found another document
in "<unicode string>", line 4, column 1:
---
^

The file looks like this

---
layout: blog
title: Hello World
date: 2020-04-15T20:17:37.845Z
---
Test

Figured it out! Now that I’m getting this CMS right, I’ll create my own blog post once everything is styled. Following the OP’s lead I created my own clean up to my freeze.py:

freezer = Freezer(app)

if __name__ == '__main__':
    for page in os.listdir('pages'):
        # Read in the file
        with open('{}/pages/{}'.format(os.getcwd(), page), 'r') as f:
            filedata = f.readlines()

        with open('{}/pages/{}'.format(os.getcwd(), page), 'w') as f:
            for n, line in enumerate(filedata):
                # Need to remove the first line if it's ---, but if we already cleaned it up we shouldn't
                if n == 0 and line == '---':
                    continue
                # All --- get replaced with empty spaces but we keep the \n
                line = line.replace('---', '')

                f.write(line)
    freezer.freeze()

yay! thanks for sharing your solution!

Actually I spoke too soon and I’ll take a minute for the full fix.

This workaround works: I create a new blog article. Netlify deploys it. The ‘—’ delimiters are removed on deploy when python freeze.py is run and the frontend picks it up. Great! However, those delimiters are necessary for Netlify CMS and now I see blank blog entries without titles in the Netlify CMS UI. The body contains the raw Markdown and is not formatted.

I think the answer is to have two folders one for Netlify CMS and one for Flask-FlatPages, which I’ll do for fun, but this appears to be quite the workaround for something that should work out of the box.

Why can’t Netlify CMS support python-markdown again? Or is there an equivalent?

As far as I can tell, Netlify CMS and Flask-FlatPages have conflicting requirements for post metadata. I got lucky when I found a post metadata format that worked for both Pelican and Netlify CMS. You might not be as lucky.

1 Like

I wonder if this would work:

  1. In your Netlify CMS collection configuration, set your frontmatter_delimiter to ### (docs)
  2. Change the delimiters in your source files from --- to ###
  3. Add a blank line after the closing ### delimiter

I think the YAML parser in Flask-FlatPages would treat the ### lines as comments.

I tried that and got a similar YAML error :-/ But creating a separate Flask-FlatPages /pages and Netlify CMS /posts folder solved the immediate problem:

if __name__ == '__main__':
    pages = os.listdir('pages')
    for post in os.listdir('posts'):
        if post not in pages:
            # Read in the file
            with open('{}/posts/{}'.format(os.getcwd(), post), 'r') as f:
                filedata = f.readlines()

            with open('{}/pages/{}'.format(os.getcwd(), post), 'w') as f:
                for n, line in enumerate(filedata):
                    # Need to remove the first line if it's ---
                    if n == 0:
                        line = line.replace('---\n', '')
                    # All --- get replaced with empty spaces but we keep the \n
                    line = line.replace('---', '')
                    f.write(line)
    freezer.freeze()