Repository navigation
The finding-block grammar is an undocumented contract: consumers parse observed behaviour and every upgrade is a re-verification #101
Description
Anything consuming the artifact programmatically has to parse:
- the
[finding:<id>|<sev>],[agree:…],[dispute:…],[observation]and[no-findings]markers - the
— via,— risk:,— evidence:and— atfields - the payload boundary, which is everything after the last
## Reviewheading — the last, because earlier ones occur inside the quoted description and the diff
None of that is documented as a stable contract. It is all observed behaviour, so a consumer cannot distinguish a deliberate format change from an incidental one, and every version bump becomes a manual re-verification: diff two plugin caches, grep for the marker and field names, then read real output to confirm nothing moved.
That is workable but backwards — the burden of detecting a breaking change sits with every consumer independently, and a consumer that misses one degrades silently rather than failing. A renamed field does not error; it just stops being recognised, and the content it carried renders as an unlabelled paragraph.
Proposal
Either of:
- Document the block grammar and the payload boundary as a versioned contract, so a change to either is a deliberate, visible event.
- Emit a machine-readable sidecar (JSON) alongside the markdown, so consumers stop parsing prose entirely.
(2) is the more robust of the two, and would also remove the ## Review last-heading rule, which is the sharpest edge here — taking the first heading publishes the scaffolding the boundary exists to strip.
Reproduce
Upgrade across two minor versions and diff the emitted marker and field names. There is no changelog entry or schema to consult, so the only way to know whether anything changed is to compare output.
Activity
kevin-agrology commentedon Sep 1, 2026
Some field evidence for this, from three version bumps in three days. Each one needed the same manual work — diff two plugin caches, extract the block markers and field names, compare — and only one of the three actually needed it.
1.27.0 → 1.28.0 added a new block marker,
> [resolved:<id>]. That is a contract change, and a consumer that did not diff would not have failed loudly: an unrecognised>line is a continuation of the preceding block, so the resolved note renders as part of the finding it resolves, and that finding keeps its old state. No error, no log line, a review that reads as blocked on something already fixed. It was caught by reading commit messages on the way to something else.1.28.0 → 1.28.1 changed no contract surface whatsoever — the markers and field names are byte-identical, and the diff is confined to
channel-check. But that was only knowable after doing the full comparison.That asymmetry is the cost this issue describes. Both upgrades required identical manual verification; one was a breaking-shaped change and the other was not, and nothing distinguished them in advance. The work scales with release cadence rather than with actual contract churn.
A changelog entry that names contract-affecting changes explicitly would collapse the common case to a read. Something as small as a marked section listing new or renamed block markers, new or renamed
— field:names, and any change to the payload boundary rule would be enough — a consumer could then diff only when that section is non-empty, and trust a patch release that says nothing.That is strictly less work than the JSON sidecar proposed above, and would capture most of the benefit for consumers who only need to know whether something moved.