Custom Writer: two blank lines above Markdown headers

[swarn]

This seems like it should be easy. Given the input:

# Top Level 1
Text

## Sub heading
Text

## Sub heading
Text

The command pandoc -t commonmark_x produces:

# Top Level 1

Text

## Sub heading

Text

## Sub heading

Text

And I would like to produce nearly the same thing, but with two blank lines above each section heading:

# Top Level 1

Text


## Sub heading

Text


## Sub heading

Text

Examples make me think this should be easy, with a custom Lua writer something like:

function Writer(doc, opts)
  local filter = {
    Header = function(header)
      -- something with pandoc.layout.blanklines(2)
      return header
    end,
  }
  return pandoc.write(doc:walk(filter), "commonmark_x", opts)
end

Template = pandoc.template.default "commonmark_x"

But I'm not quite figuring out how to make it click. The filter is dealing with Blocks, the writer with Docs, and I'm sure I'm missing something simple to make it work.

[swarn]

Progress:

Writer = pandoc.scaffolding.Writer

Writer.Block.Header = function (header)
  return {
    pandoc.layout.blanklines(2),
    string.rep("#", header.level) .. ' ',
    Writer.Inlines(header.content)
  }
end

Writer.Block.Para = function (para)
  return {Writer.Inlines(para.content), pandoc.layout.blankline}
end

Writer.Inline.Str = function (str)
  return str.text
end

Writer.Inline.Space = function (space)
  return pandoc.layout.space
end

This works well enough for the example text. But it would require defining functions for all AST nodes. In an OO paradigm, I would subclass a "CommonMarkWriter" visitor, then override the "Header" method (while calling the original method after adding blank lines). I'm not sure how to proceed with the lua API here.

[jgm]

Looking at doclayout, it does not exhibit this behavior, which makes me quite puzzled.

ghci> render (Just 20) (text "a" $$ text "\n\n" $$ text "b" :: Doc Text)
"a\n\n\nb"

[jgm]

More to the point, as literal is used to render the raw block:

ghci> render Nothing (text "a" $$ literal "\n\n\n" $$ text "b")
"a\n\n\n\nb"

[jgm]

Aha:

ghci> putStrLn $ render Nothing (text "a" $+$ literal "\n\n\n\n\n" $+$ text "b")
a

b

In the Doc structure BlankLines 1, NewLine, NewLine the newlines are being collapsed to ensure that there is just one blank line. This is how doclayout is intended to work, but there should be some way to suppress it.

[jgm]

OK, I've pushed a commit that should allow these initial/final newlines in markdown raw blocks to be preserved.

[jgm]

You may still need to put one more newline than you think in the raw block, because of stripping that happens in Text.Pandoc.Builder.rawBlock.

[swarn]

Many thanks! With the newest commit, this filter does the job for me:

function Header(elem)
  return {
    pandoc.RawBlock("markdown", "\n\n\n"),
    elem,
  }
end

I'm looking forward to whatever comes out of your conversations about extending writers. It would be fun to write something like:

Writer = pandoc.writer.clone "commonmark_x"

local orig_header = Writer.Block.Header

Writer.Block.Header = function(header)
  return {
    pandoc.layout.blanklines(2),
    orig_header(header),
  }
end

[jgm]

Agreed. The problem is that the writers generally have a state, and this isn't encapsulated in the same way it is in OOP programs, so I don't think it's entirely straightforward to make custom writers work the way you want them to....at least not without major architectural changes.

[swarn]

It's been 15 years since I've written non-trivial Haskell code. I can only guess at the challenge of making your State monad or catamorphism visible and modifiable through Lua. It'll be cool if you can pull it off!