Allow line and document termination to be equivalent at | when parsing pipe table rows

[criloz]

problem

based on the behavior of the JS implementation, this is considered a table

| lklk  | lklkk |

https://djot.net/playground/?text=%7C+lklk++%7C+lklkk+%7C%0A&sourcepos=false

but this is not

| lklk  | lklkk 

https://djot.net/playground/?text=%7C+lklk++%7C+lklkk+%7C%0A&sourcepos=false

I am currently writing a stream parser for djot and the table spec make it unnecessary complex to do it without buffering.

The parser needs to wait until the line ends to process the whole line as a row or as a paragraph, and weird things can happen with texts like this | lklk *|* lklkk and | lklk *|* lklkk|

solution

Would not be better to just consider the line and document termination events as valid, to parse the line as a table row when it starts with |.

It looks that it does not create any kind of conflicts with any other rules and will allow writing less complex, more secure and faster parsers.

[jgm]

I think I may have been reserving the possibility of using the syntax

| blah blah blah
| blah blah

for something else -- cf. reST (and pandoc markdown) "line blocks." That's the only drawback I see to your proposal; it would prevent us from using this distinctive syntax for something.

[criloz]

@jgm Ahh, I see, that is a good reason, but if is not imperative to have the same syntax from Pandoc a new kind of token could be introduced like |: or |> and still maintain the simplicity of the parser, it will reduce ambiguity and the need to have parallel states, buffering or backtracking

|> The limerick packs laughs anatomical
|> In space that is quite economical.
|>    But the good ones I've seen
|>    So seldom are clean
|> And the clean ones so seldom are comical

edit

a better token could be |> because |: will also create an ambiguity with table

[Omikhleia]

Just for the cross-reference: #29