✅ Normalization and Semantic Line Breaks

[kiil]

A problem with iwe's normalization is this

A markdown file will change from

# Test

Hej med dig.
Du er sej.

to

# Test

Hej med dig.Du er sej.

on a simple save in the editor or on iwe normalize from the cli.

This should not happen. Because of Semantic Line Breaks. People use this for keeping markdown files and other plain text source files easier to both read AND edit fast on a line by line basis.

(and even if it should happen, at least keep spaces between the joined lines)

[gimalay]

This one is tricky. IWE core is build in a way that it works with Markdown elements, not the text. In markdown there is no such element as semantic line break.

[kiil]

Thanks for explaining :)

Would it at least be possible to join the lines with a space between the sentences?

(It is rather easy to split the sentences into lines again in the editor, so that is not the main problem here.)

[gimalay]

I'm thinking about adding more auto-formatting, for example automatically added a spaces after .!? symbols if there is none and replace two spaces with one, etc. This should partially solve the problem you described here.

[kiil]

Sounds good!

I think normally this:

en
to
tre

Would become this

en to tre

not this

entotre

At least that seems to be the case in html and markdown, and as far as I know it is that way for lines in many languages when lines are joined in an editor or for display :)

[gimalay]

I just added a fix for this.

en
to
tre

Would become this

en to tre

Thanks for your input! I hope this helps.

[kiil]

Thank you very much. I think this is the right choice, the "standard", and what most would expect. :)

[kiil]

@gimalay

I've played around with

preserve_line_breaks: Keep hard line breaks instead of dropping them (default: false). Recognizes two trailing spaces ( \n) and backslash (\\n) in the source and emits them in the configured line_break_style on output.

I think simply \n (soft break) should also be recognized and then there could be a matching line_break_style for the output so lines in parapgraphs would stay the same and "survive" normalize / auto format

That would enable semantic line breaks and be in spirit with the idea that the source of markdown should be easily readable.

Or maybe easier: Another option semantic_line_breaks that simply omitted the joining of consecutive lines on normalize / auto format and left the source as is for consecutive lines with soft breaks in paragraphs.

A question - this PR sought to address this issue? #204 or did that just change from joining directly to joining with a space?

[gimalay]

Hey @kiil ! This makes sense. Here is the change #328

#204 just changed "join directly" → "join with a space."

I guess, I've missed this the first time you requested this. There is a SoftBreak support in the parser and the SoftBreaks can be preserved in the output.

Please let me know if this works for you.

[kiil]

@gimalay Thanks a lot. This new option is extremely helpfulk to my workflow. Thanks for adding it!

[gimalay]

No problem, please let me know if there is any other improvements we can make.