Raw Node Type

[stellarpower]

Hi,

I'm not consuming Comrak directly but using it through Commonmarker in ruby. I took a look at the node type enum before though and didn't see a clear candidate.

Is it possible to add a node to the document tree that's completely raw text, never to be escaped? If not, would it be possible to add this as a type, and then disallow the adding of children and under no circumstances perform any escaping or processing when rendering?

I'm taking content from a visual editor and performing some processing before handing it off to Zola to generate a static site. Zola makes use of shortcodes, which are in syntax a bit like jinja/liquid. Unfortuately these sections therefore aren't markdown, and comrak is being strictly-compliant and escaping the underscores in some URLs that are within quotes. So right now I think the only sane option I have is to insert some UUIDs and just string-replace after I have written the document back out, which is horrible, but safe at this point.

Thanks

[wakairo]

Hi,

I am another person who is a user of Commonmarker and request support for a Raw Node Type.

I have two use cases where this feature would be beneficial:

1. Embedding Content from URLs

When a link in Markdown points to embeddable content, I would like to convert it into embedded HTML.

For example, given the following YouTube link:

https://www.youtube.com/watch?v=lJIrF4YjHfQ

I would like to transform it into:

<iframe width="560" height="315" src="https://www.youtube.com/embed/lJIrF4YjHfQ?si=Faw28QGM0TGFxypA" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

2. Adding a Copy Button for Code Blocks

Similar to GitHub, I would like to add a "Copy" button to Markdown code blocks, as shown below.
To achieve this, I need the ability to insert a button or image HTML element into the Markdown node tree.

[kivikakk]

Is it possible to add a node to the document tree that's completely raw text, never to be escaped? If not, would it be possible to add this as a type, and then disallow the adding of children and under no circumstances perform any escaping or processing when rendering?

I think HtmlInline and HtmlBlock might fit the bill, as long as you don't have options.render.escape enabled, do have options.render.unsafe_ enabled, and aren't using options.extension.tagfilter. Given these settings, these nodes' contents are rendered verbatim in HTML.

When a link in Markdown points to embeddable content, I would like to convert it into embedded HTML.

I don't think a raw HTML type really helps you here? You can find the generated link (or raw text, if not using options.extension.autolink) and insert a HtmlBlock as you like.

Similar to GitHub, I would like to add a "Copy" button to Markdown code blocks, as shown below.
To achieve this, I need the ability to insert a button or image HTML element into the Markdown node tree.

This is really not within Comrak's remit; that kind of thing doesn't really belong in your source document.

[stellarpower]

Given these settings, these nodes' contents are rendered verbatim in HTML.

It's a while ago since I reported this, so my memory's not perfect. But I think I would have tried this. I tried a few different methods for a raw type, and either I wasn't able to edit the settings (as the package I'm using directly was doing this) or it didn't quite work, and eventually gave up and did it using regexes after writing back out again. I'll take a look again if I come back to this and see what I did. Also, for reference, I'm not rendering to HTML - just using the package to manipulate the document tree. So it'd need to be something that can render out in markdown again, not only for HTML.

Thanks!

[kivikakk]

Hm. Lessee! Here's a quite janky example using HtmlBlock, but it does show what you want, I think. (I copied a few shortcode examples from the Zola docs page on them.)

fn main() {
    use comrak::arena_tree::Node;
    use comrak::nodes::{Ast, NodeHtmlBlock, NodeValue};
    use comrak::{format_commonmark, parse_document, Arena, Options};
    use std::cell::RefCell;

    let doc = "_Hello!_\n\n(RAW)\n\n**Bye!**";

    let example = r#"
<div {% if class %}class="{{class}}"{% endif %}>
    <iframe
        src="https://www.youtube.com/embed/{{id}}{% if autoplay %}?autoplay=1{% endif %}"
        webkitallowfullscreen
        mozallowfullscreen
        allowfullscreen>
    </iframe>
</div>

{% set data = load_data(path=path) -%}
{% for book in data.books %}
### {{ book.title }}

{{ book.description | safe }}
{% endfor %}
    "#;

    let arena = Arena::new();
    let root = parse_document(&arena, doc, &Options::default());

    for node in root.descendants() {
        if let NodeValue::Text(ref text) = node.data.borrow_mut().value {
            if text == "(RAW)" {
                let raw_ast = Ast::new(
                    NodeValue::HtmlBlock(NodeHtmlBlock {
                        block_type: 0, // only used during embedded HTML parse
                        literal: example.to_string(),
                    }),
                    (0, 0).into(),
                );
                let raw_node = arena.alloc(Node::new(RefCell::new(raw_ast)));
                // Replace the containing paragraph node with the HtmlBlock.
                node.parent().unwrap().insert_after(raw_node);
                node.parent().unwrap().detach();
                break;
            }
        }
    }

    let mut buf = vec![];
    let mut options = Options::default();
    options.render.unsafe_ = true;
    format_commonmark(root, &options, &mut buf).unwrap();
    let html = String::from_utf8(buf).unwrap();

    println!("{}", html);
}

Output:

*Hello\!*


<div {% if class %}class="{{class}}"{% endif %}>
    <iframe
        src="https://www.youtube.com/embed/{{id}}{% if autoplay %}?autoplay=1{% endif %}"
        webkitallowfullscreen
        mozallowfullscreen
        allowfullscreen>
    </iframe>
</div>

{% set data = load_data(path=path) -%}
{% for book in data.books %}
### {{ book.title }}

{{ book.description | safe }}
{% endfor %}
    

**Bye\!**

[wakairo]

@kivikakk

Thank you for the information and the code example.

I'm developing a web application, and I need to safely escape Markdown text provided by users. However, the system should be able to manipulate the Markdown and insert nodes without escaping them.

I believe that if options.render.unsafe_ is enabled, the nodes inserted by the system are escaped, and if options.render.unsafe_ is disabled, user input is not escaped.

Is there a good solution in the current version of Comrak? Could adding a raw node type be a viable solution?

[kivikakk]

@wakairo @stellarpower The more I think about this, the more I think it's reasonable and see that it can't be achieved by any other means right now.

Would either of you two be up to attempting a PR? I'm happy to guide, I just don't have the time or energy to put one together myself right now.

[stellarpower]

Hey, thanks for the responses. Afraid I've not had time to look, it was a work project and I've not had to go back to it just yet. Also I'm consuming through ruby, so would be a bit longer to try your snippet as I'd need to use the raw package.

But yes, am lacking in time and energy myself. However, if I end up needing it significantly, I don't mind in principle having a crack at it. And then some guidance would definitely be helpful as I've only dipped my toes onto rust this far. Hopefully shouldn't be too hard if it's raw, as opposed to needing formatting.

Thanks!

[wakairo]

@kivikakk I have submitted PR #511. Could you please take a look?

[kivikakk]

That PR has been merged! :) We now have NodeValue::Raw.