Block Expressions throw exception if Markdown syntax detected.

[martinhalford]

Describe the bug
Block Expressions throw an error if markdown syntax detected within block.

To Reproduce
Consider the following grammar...

{{#if rebateEntitled}} 
The agent will be, or is likely to be, receiving a rebate or rebates.
{{else}} 
The agent will not be, or is not likely to be, entitled to any rebate in respect of\
...any outgoings; or\
...any prepayments made by a person engaging or appointing the agent (the client) in respect of any intended expenditure by the agent on the client's behalf; or\
...any payments made by the client to another person in respect of the work.
{{/if}}

The above renders fine.

Now consider the following grammar...

This is a contract
{{#if rebateEntitled}} 
The agent will be, or is likely to be, receiving a rebate or rebates.
{{else}} 
The agent will not be, or is not likely to be, entitled to any rebate in respect of\
- any outgoings; or\
- any prepayments made by a person engaging or appointing the agent (the client) in respect of any intended expenditure by the agent on the client's behalf; or\
- any payments made by the client to another person in respect of the work.
{{/if}}
This is more text

The above fails with error: Cannot read property 'length' of undefined due to the - chars which are interpreted as unordered list syntax.

Now consider the following grammar...

{{#if rebateEntitled}} 
The agent will be, or is likely to be, receiving a rebate or rebates.
{{else}} 
The agent will not be, or is not likely to be, entitled to any rebate in respect of\

...any outgoings; or\
...any prepayments made by a person engaging or appointing the agent (the client) in respect of any intended expenditure by the agent on the client's behalf; or\
...any payments made by the client to another person in respect of the work.
{{/if}}

The above fails with error: Cannot read property 'length' of undefined due to the blank line within the {{else}} block.

Expected behavior
Expect the Conditional block to consider all chars, that are not part of the Block syntax, be treated as plain text and no error be thrown.

Similarly, expect the new / blank line to be treated as plain text and/or converted to a hardline break (e.g. \) and not throw an error.

Version
Cicero v0.21.5

Additional context
ifelse.zip

[jeromesimeon]

Thanks again for the detailed report, @martinhalford. Just a couple of clarifications for the purpose of issue triaging this:

1. some markdown is allowed inside conditionals (and join/optional/etc), but is limited to commonmark inlines (i.e., strong/emph/code/html inlines). If the text spans multiple commonmark blocks (i.e., multiple paragraphs, lists, block quotes) the template conditionals/optionals etc will not be detected by the parser.

For instance, this is fine:

This is a markdown {{#if foo}}(with some _conditional_ text, with some `code` and <html/> in it){{/if}} inside a **paragraph**.

but this isn't:

This is a markdown {{#if foo}}(with some _conditional_ text).

(also with some `code` and <html/> in it){{/if}} across two **paragraphs**.

This behaviour is consistent with other inline markup in commonmark. Here is an example with emphasis:

Spec reference for the distinction between commonmark blocks and commonmark inlines: https://spec.commonmark.org/0.29/#blocks-and-inlines

[jeromesimeon]

2. The behaviour is expected to be similar to the one for the inlines markup (like emphasis) i.e., it should gracefully fall back to text parsing (as with emphasis in the example above) rather than raise an error. This is meant to remain consistent with the very first line in the commonmark spec:

Any sequence of characters is a valid CommonMark document.

So this issue reports a genuine bug. The following error should not be thrown: error: Cannot read property 'length' of undefined.

[jeromesimeon]

3. Note that the reason for the error, isn't quite due to wrongly parsing the block-level markdown, but likely due to faulty logic handling closing tags (or missing closing tag since {{/if}} isn't present), within a single block. Likely fixes should happen in the following part of the markdown-it-cicero and markdown-it-template plugins:

• https://github.com/accordproject/markdown-transform/blob/master/packages/markdown-it-template/lib/template_inline.js
• https://github.com/accordproject/markdown-transform/blob/master/packages/markdown-it-cicero/lib/cicero_inline.js

[martinhalford]

Thanks @jeromesimeon - I kinda expected that the new line would throw an error and, indeed, this can be mitigated by added a \ to the paragraph, and that then works fine. If the - and other unsupported markdown syntax could revert to plain text then that would be great!

[jeromesimeon]

A few minimal tests for this issue, for convenience.

mini.zip

[jeromesimeon]

More tests for this based on the issuer's example
inlines.zip

With the fix in #341

• closing inline blocks (e.g., {{/with}}) without a matching opening inline block (e.g., {{#with}} will not raise an error anymore but be treated as text
• opening inline blocks (e.g., {{#with}) without a matching closing inline blocks (e.g., {{/with}}) will not raise an error anymore but are automatically closed at the end of a paragraph/heading etc.

This should result a proper / better documented semantics for the various templates grammars from the example.

For instance:

This is a contract
{{#if rebateEntitled}} 
The agent will be, or is likely to be, receiving a rebate or rebates.
{{else}} 
The agent will not be, or is not likely to be, entitled to any rebate in respect of\
- any outgoings; or\
- any prepayments made by a person engaging or appointing the agent (the client) in respect of any intended expenditure by the agent on the client's behalf; or\
- any payments made by the client to another person in respect of the work.
{{/if}}
This is more text

Is effectively treated as:

This is a contract
{{#if rebateEntitled}} 
The agent will be, or is likely to be, receiving a rebate or rebates.
{{else}} 
The agent will not be, or is not likely to be, entitled to any rebate in respect of\{{/if}}

- any outgoings; or\
- any prepayments made by a person engaging or appointing the agent (the client) in respect of any intended expenditure by the agent on the client's behalf; or\
- any payments made by the client to another person in respect of the work.
{{/if}}
This is more text

With the first {{#if}} closed automatically at the end of the paragraph before the beginning of the list, and the last {{/if}} being treated as text since it does not have a matching {{#if}}

[jeromesimeon]

Note: The rule for {{#if}} being only an inline (i.e., it cannot span multiple markdown container blocks) still applies.

[jeromesimeon]

Example of the draft command on the third template before #316:

bash-3.2$ markus transform --from data --to markdown_cicero --template grammar3.tem.md --model model.cto --input dataa.json 
3:32:42 PM - error: Cannot read property 'length' of undefined

Example of the draft command on the third template after #316:

bash-3.2$ ~/git/markdown-transform/packages/markdown-cli/index.js transform --from data --to markdown_cicero --template grammar3.tem.md --model model.cto --input dataa.json 
<data --to markdown_cicero --template grammar3.tem.md --model model.cto --input dataa.json 
3:31:23 PM - info: 
The agent will be, or is likely to be, receiving a rebate or rebates.
-  any outgoings; or\
-  any prepayments made by a person engaging or appointing the agent (the client) in respect of any intended expenditure by the agent on the client's behalf; or\
-  any payments made by the client to another person in respect of the work.{{/if}}
bash-3.2$ ~/git/markdown-transform/packages/markdown-cli/index.js transform --from data --to markdown_cicero --template grammar3.tem.md --model model.cto --input datab.json 
<data --to markdown_cicero --template grammar3.tem.md --model model.cto --input datab.json 
3:31:31 PM - info: 
The agent will not be, or is not likely to be, entitled to any rebate in respect of\
-  any outgoings; or\
-  any prepayments made by a person engaging or appointing the agent (the client) in respect of any intended expenditure by the agent on the client's behalf; or\
-  any payments made by the client to another person in respect of the work.{{/if}}