Are Marked.Lexer and Marked.Parser meant to inherit the options from Marked.use()?

[calculuschild]

This might be a dumb question. I'm trying to split the render process on my app into separate Marked.Lexer.lex and Marked.Parser.parse steps. For some reason, the extensions and options I load via Marked.use only seem to apply to the Lexer. I can get a good Token tree, but the Parser crashes due to not recognizing that marked-gfm-heading-id is being used, for example.

On that note, am I missing a clear way to export the options object from Marked? If I need to just apply all the options to the Lexer and Parser separately, it would be nice to just read out the object to reuse later. Marked.options gives me this function instead of the options object:

https://github.com/markedjs/marked/blob/ba3b7e4be7b02699b44d54a68df74c9f8ac7ad4b/src/marked.ts#L45C1-L51C3

[UziTech]

Marked.lexer and Marked.parser add the options for an instance. You can get the options with marked.defaults. For legacy reasons marked.options is the same function as marked.setOptions

marked/src/Instance.ts

Lines 233 to 239 in ba3b7e4

	lexer(src: string, options?: MarkedOptions) {
	return _Lexer.lex(src, options ?? this.defaults);
	}
	parser(tokens: Token[], options?: MarkedOptions) {
	return _Parser.parse(tokens, options ?? this.defaults);
	}

[calculuschild]

Ah, I see what was going on. marked-gfm-heading-id uses the new Preprocess and Postprocess hooks, so when I split it up into separate lex and parse steps, that preprocessor wasn't being run.

Do you mind if I add to the docs some instructions on how to split up marked.parse() into each step? I don't think it's quite obvious that to replicate the complete parsing pipeline you need to run something like:

		const opts = Marked.defaults;

		srcText = opts.hooks.preprocess(srcText);         // 1) Preprocess String
		const tokens = Marked.lexer(srcText, opts);       // 2) String to Token Tree
		Marked.walkTokens(tokens, opts.walkTokens);       // 3) Walk Tokens
		const html = Marked.parser(tokens, opts);         // 4) Token Tree to HTML
		return opts.hooks.postprocess(html);              // 5) Postprocess HTML

especially where walkTokens needs to be passed specifically the options.walkTokens property.

Eventually, it might be nice to expose the different hooks directly so they can be called without having to dig into the options object. And have walkTokens automatically use the options from the instance by default like lexer and parser do.

[UziTech]

Ya that sounds cool. When I created the Marked instance I tried getting everything to use the default options but some things are weird to accommodate global marked having global options and local options.

For example Tokenizer.rules is set by the Lexer not in the Tokenizer constructor.

It would all be a lot easier if marked didn't have global options.