Blank line paragraph preservation#9872
Conversation
✅ Deploy Preview for tiddlywiki-previews ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
📊 Build Size Comparison:
|
| Branch | Size |
|---|---|
| Base (master) | 2491.8 KB |
| PR | 2495.7 KB |
Diff: ⬆️ Increase: +3.9 KB
⚠️ Change Note Status
This PR appears to contain code changes but doesn't include a change note.
Please add a change note by creating a .tid file in editions/tw5.com/tiddlers/releasenotes/<version>/
📚 Documentation: Release Notes and Changes
💡 Note: If this is a documentation-only change, you can ignore this message.
|
Confirmed: linonetwo has already signed the Contributor License Agreement (see contributing.md) |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR adds an opt-in (and configurable) way to preserve extra blank lines in TiddlyWiki wikitext by representing them as empty paragraph blocks, and ensures they round-trip through serialization and display.
Changes:
- Introduces
preserveBlankLinesparser option (and$:/config/Parser/PreserveBlankLines) to emit explicitblanklineparse tree nodes for additional blank lines. - Updates wikitext serialization to round-trip
blanklinenodes back into the correct newline sequences. - Adds theme styling and a dedicated test spec to validate parsing/serialization behavior.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| themes/tiddlywiki/vanilla/base.tid | Adds CSS to ensure empty blankline paragraphs have visible height |
| plugins/tiddlywiki/wikitext-serialize/utils/parsetree.js | Serializes blankline parse tree nodes back into newline text |
| editions/test/tiddlers/tests/test-wikitext-blanklines.js | Adds tests covering parsing + serialization for preserved blank lines |
| core/modules/wiki.js | Plumbs preserveBlankLines through wiki.parseText() into parsers |
| core/modules/parsers/wikiparser/wikiparser.js | Implements blank line node generation and integrates it into block parsing |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| WikiParser.prototype.makeBlankLineBlocks = function(start,whitespace,options) { | ||
| options = options || {}; | ||
| var newlineMatches = whitespace.match(/\r?\n/g), | ||
| newlineCount = newlineMatches ? newlineMatches.length : 0, | ||
| blankLineCount = options.leading ? Math.max(0,newlineCount - 1) : Math.max(0,newlineCount - 2), | ||
| blankLineBlocks = []; | ||
| for(var index = 0; index < blankLineCount; index++) { | ||
| blankLineBlocks.push({ | ||
| type: "element", | ||
| tag: "p", | ||
| attributes: { | ||
| class: {name: "class", type: "string", value: "tc-blankline"} | ||
| }, | ||
| orderedAttributes: [ | ||
| {name: "class", type: "string", value: "tc-blankline"} | ||
| ], | ||
| children: [], | ||
| start: start, | ||
| end: start, | ||
| rule: "blankline", | ||
| isLeadingBlankLine: !!options.leading && index === 0 | ||
| }); | ||
| } |
| var newlineMatches = whitespace.match(/\r?\n/g), | ||
| newlineCount = newlineMatches ? newlineMatches.length : 0, |
| it("should preserve extra blank lines as empty paragraphs", function() { | ||
| expect(paragraphCount("A\n\n\nB")).toBe(3); | ||
| expect(parse("A\n\n\nB")[1].attributes.class.value).toBe("tc-blankline"); | ||
| expect(serialize("A\n\n\nB")).toBe("A\n\n\nB\n\n"); | ||
| expect(paragraphCount("A\n\n\n\nB")).toBe(4); | ||
| expect(serialize("A\n\n\n\nB")).toBe("A\n\n\n\nB\n\n"); | ||
| }); |
linonetwo
changed the title
|
Thanks @linonetwo I think this approach looks promising. |
|
If this way is valid, I would expect remove the If I make multiple-line support enabled by default, Will this break some backward-compatility? |
|
Do you have the same problem with HTML comments? |
|
I will take a look. Maybe we should make it CST instead of AST. Keep all details in the wikitext so we can preserve it at-is. |
|
hmmm, I think CST is overkill on the library side, if you think about Lezer ... IMO we would need 3 things.
IMO that should give us all the info we need. On the other hand. I personally would like to have a "TW prettyprinter" active all the time, which uses "safe" TW syntax. ... But that's a completely different story. |
|
The pattern is called: "trivia-enriched tree". It seems Roslyn (the C# compiler) does almost exactly this: the semantic tree stays clean, and every token carries leading/trailing trivia (whitespace and comments). |
Summary
closes #9871
This PR introduces a
blanklineparse tree rule when preservation is enabled:A\n\nBremains two paragraphs separated by the normal paragraph separatorA\n\n\nBbecomesparseblock, blankline, parseblockA\n\n\n\nBbecomesparseblock, blankline, blankline, parseblockIt also handles extra blank lines after non-paragraph block rules. This matters for content such as lists followed by macros or paragraphs, where the block rule may consume trailing whitespace before the outer block parser sees it.
What changed
preserveBlankLinesparser option support and$:/config/Parser/PreserveBlankLinesconfig supportpreserveBlankLinesthroughwiki.parseText()to the parserblanklineparse tree nodes withclass="tc-blankline"blanklinenodes back to the extra newline they representCompatibility
The behavior is opt-in for now:
{ preserveBlankLines: true }towiki.parseText(); or$:/config/Parser/PreserveBlankLinestoyesWith the option disabled, existing behavior is preserved: extra blank lines are ignored as before.
Future path to remove the option
I don't really want the
preserveBlankLinesoption, but directly enable blank line support breaks many test, so I have to add it now, until we decide to enable it globally.If the project decides that preserving extra blank lines should become the default wikitext behavior, the feature option can be removed by:
core/modules/parsers/wikiparser/wikiparser.jsunconditional$:/config/Parser/PreserveBlankLinescheck and thepreserveBlankLinesparser optionpreserveBlankLinespass-through fromcore/modules/wiki.jsblanklineserializer support, because parse trees may still containblanklinenodesTests to update if the option is removed:
editions/test/tiddlers/tests/test-wikitext-blanklines.jsA\n\n\nBto collapse should be updated to expect ablanklineempty paragraph instead{ preserveBlankLines: true }parser calls once the parser behavior is default