Skip to content

feat: HTML parser#57

Merged
KnightNiwrem merged 12 commits into
grammyjs:masterfrom
KnightNiwrem:feat/html-stream-parser-v2
Mar 13, 2026
Merged

feat: HTML parser#57
KnightNiwrem merged 12 commits into
grammyjs:masterfrom
KnightNiwrem:feat/html-stream-parser-v2

Conversation

@KnightNiwrem

Copy link
Copy Markdown
Member

Note: Tests and select utility methods written by LLM
Note: Parsing methods are handwritten and deliberately rolled out (i.e. not refactored for DRY) at this moment - i.e. putting all the state transitions on paper first

Copilot AI review requested due to automatic review settings February 18, 2026 09:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces a new streaming HTML parser (HTMLStreamParser) that converts Telegram HTML-style text into FormattedString instances, along with initial tests and a specification document.

Changes:

  • Added HTMLStreamParser state-machine implementation for supported tags/entities and Telegram entity mapping.
  • Added test coverage for several core tag mappings and chunk-boundary streaming behavior.
  • Exported the new parser from the library entrypoint and added a spec document describing intended completion criteria.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.

File Description
src/stream-html-to-format.ts New streaming HTML parser implementation and entity mapping logic.
test/stream-html-to-format.test.ts New tests validating core tag/entity mappings and streamed chunk handling.
src/mod.ts Re-exports the new parser module.
specs.md Adds a detailed refactor/specification document for the parser behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/stream-html-to-format.ts Outdated
Comment thread test/stream-html-to-format.test.ts
Comment thread src/stream-html-to-format.ts
Comment thread src/stream-html-to-format.ts
Comment thread src/stream-html-to-format.ts Outdated
Comment thread src/stream-html-to-format.ts
Comment thread src/stream-html-to-format.ts Outdated
Comment thread src/stream-html-to-format.ts Outdated
Comment thread src/stream-html-to-format.ts Outdated
KnightNiwrem and others added 5 commits February 18, 2026 18:22
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@KnorpelSenf

Copy link
Copy Markdown
Member

What's the deal here? Wanna instruct LLMs to output HTML and then use this to stream formatted drafts?

@KnightNiwrem

Copy link
Copy Markdown
Member Author

What's the deal here? Wanna instruct LLMs to output HTML and then use this to stream formatted drafts?

Don't really have LLMs as an intended audience for this, though it might coincidentally work (i.e. this parser is somewhat lenient on incomplete HTML strings). The main (and original) audience would be i18n users. Instead of wrangling with HTML parse_mode, we want to locally convert HTML strings (which i18n users are forced to use as they can't do template literals in .ftl) into text+entities.

@KnightNiwrem

Copy link
Copy Markdown
Member Author

Only works coincidentally because of the chosen architecture of the parser

const s = new HTMLStreamParser();

s.add('This is the start of ');
console.log(s.toFormattedString());

s.add('some <b>bolded content ');
console.log(s.toFormattedString());

s.add('that later</b> end with plain text');
console.log(s.toFormattedString());

=======

parse-mode % deno run src/stream-html-to-format.ts 

FormattedString { rawText: "This is the start of ", rawEntities: [] }
FormattedString {
  rawText: "This is the start of some bolded content ",
  rawEntities: []
}
FormattedString {
  rawText: "This is the start of some bolded content that later end with plain text",
  rawEntities: [ { type: "bold", offset: 26, length: 25 } ]
}

@KnorpelSenf

Copy link
Copy Markdown
Member

That's pretty cool!

@topcoolgames102-maker

This comment was marked as spam.

@KnightNiwrem
KnightNiwrem merged commit 1ccfc67 into grammyjs:master Mar 13, 2026
4 checks passed
@KnightNiwrem

Copy link
Copy Markdown
Member Author

Merged as unstable/experimental, but good enough for RFC from real world use

@KnightNiwrem
KnightNiwrem deleted the feat/html-stream-parser-v2 branch March 13, 2026 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants