Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 26 additions & 2 deletions .agents/skills/multi-review/protocol/multi-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,7 +289,10 @@ unrelated to those rows. Its findings still merge normally
way a provider's findings are), so a `defect` still reaches the human gate even from an
otherwise-incomplete turn.

Each round records a durable coverage line alongside the round's quarantine records:
**Round 1 records a durable coverage line** alongside the round's quarantine records — round 1,
not every round: like the symbol-check pass below, this one is derived and dispatched once per
review, because the pairs it compares mostly live in unedited text that a diff-scoped round-N copy
does not carry.

> [crossref-coverage: not applicable]
> [crossref-coverage: <M>/<M> rows verdicted]
Expand Down Expand Up @@ -358,7 +361,8 @@ An exit-1 turn is **reported, never quarantined**, for the same reason as above,
merge normally (`multi-review-star.sh merge --pass "<doc>.symcheck"`, namespaced
`symcheck-rd<N>-<id>`).

Each round records a durable coverage line alongside the round's quarantine records:
**Round 1 records a durable coverage line** alongside the round's quarantine records — round 1,
not every round: this pass is derived and dispatched once per review, not once per round.

> [symcheck-coverage: not applicable]
> [symcheck-coverage: <M>/<M> rows verdicted]
Expand All @@ -368,6 +372,26 @@ Each round records a durable coverage line alongside the round's quarantine reco
`Symbol-check pass: <M>/<M> rows verdicted`, or `Symbol-check pass: <N>/<M> rows verdicted —
INCOMPLETE`.

**Every round records what the pass therefore did not see** (#99), written by the primary at the
END of its own turn — after the edits that address that round's findings, before it touches the
marker. Because the pass runs in round 1 only, its count covers the document as the secondaries
first read it, not the ready-to-paste code the author adds afterwards while fixing findings, which
is the least-reviewed code in the change by construction. Round 1 records one too: its own fixes
are unchecked for exactly the same reason, and a review that converges at round 1 is the commonest
shape of the gap. Each round records:

> [symcheck-added: <K>]

`<K>` is the doc's current `rows` count minus round 1's `<M>` (`0` when that line said `not
applicable`), floored at `0`. The gate renders it under the coverage line — `— round 1 only: <K> more block(s) now than were checked, so at least <K> are
unchecked`, or `— round 1 only: no net change in block count since (a one-for-one swap would not
show here)`, and `— round 1 only: NO RECORD of whether blocks were added since` when no round
recorded one at all. A document where the pass never applied renders none of this.

`<K>` is a **count, not an identity diff**: rows are positional (`B<n>`) and carry line ranges that
shift on any edit above them, so a round that replaces one block with another nets zero. That blind
spot is why the zero wording names it rather than claiming nothing changed.

**Known limits of the introduces set**, in both directions. It harvests the backticked tokens from
`- Create:`/`- Produces:` lines, and a block ends only at a blank line — so a prose-heavy entry
contributes grammar fragments, and an entry that follows the block with no intervening blank is
Expand Down
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "multi-review",
"description": "Human-gated star review: Claude (the neutral primary) fans a spec/plan/PR out to N independent secondary reviewers (in-harness fable by default — disable with MULTI_REVIEW_FABLE=off — plus cross-vendor codex/gemini), adjudicates their findings, and converges — always ending at a human approval gate.",
"version": "1.35.0",
"version": "1.36.0",
"author": {
"name": "Agrology"
},
Expand Down
37 changes: 37 additions & 0 deletions commands/multi-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -895,6 +895,7 @@ re-resolve later (a mutable env var could otherwise swap providers mid-review un
`> [symcheck-coverage: <N>/<M> rows verdicted]`. Never optional: an unrecorded exit-1 turn
is byte-identical at the gate to a review where this pass was never wired up.
- **Exit 2** → a usage/infra error on YOUR side. Fix the invocation; nothing is recorded.

**Plan-lint coverage, in this same step — EVERY round, not round 1 only.** The fan-out's lint ran
this round (it blocks the fan-out until it exits 0 or 3), so record which. If it exited 3,
append `> [planlint-coverage: not applicable]` under `<doc>`'s `## Review` heading, alongside
Expand Down Expand Up @@ -1039,6 +1040,42 @@ re-resolve later (a mutable env var could otherwise swap providers mid-review un
while any resolved record lacks one, in every round: a record names an earlier-round finding by
rule, so its own round cannot be read from the id, and it is your claim regardless.

**Record what the symbol-check pass did not see — EVERY round, and here, not in the fan-out
(#99).** That pass is derived and dispatched in ROUND 1 ONLY, so it covers the document as the
secondaries first read it. Every ready-to-paste block you have just added while addressing this
round's findings is unchecked against the repository, and that is the least-reviewed code in the
change by construction. Left unrecorded, round 1's clean `<M>/<M>` is all the gate renders, and a
review that added unchecked code reads exactly like one that added none.

**This belongs at the END of your turn, not in the fan-out.** The fan-out runs BEFORE you edit
the body, so a count taken there predates this round's own fixes — and on the final round those
are precisely the blocks nobody will ever check. Take it after your last edit and before you
touch the marker, every round including round 1: round 1's fixes are unchecked too, and a review
that converges at round 1 would otherwise record nothing at all.

${CLAUDE_PLUGIN_ROOT}/scripts/multi-review-symcheck.sh rows "<doc>" | wc -l

Let `<M>` be the total from round 1's `symcheck-coverage` line — `<M>` is `0` when that line said
`not applicable`, which is the doc that shipped no blocks and has since gained some. Let `<K>` be
the count above minus `<M>`, floored at `0`. An exit 3 from `rows` means the current count is
`0`, not that the step is skipped. Append, under `<doc>`'s `## Review` heading:

> [symcheck-added: <K>]

Record it when `<K>` is `0` too — "I checked and nothing was added" and "nobody looked" are
different facts, and the gate renders them differently. **Exactly one record per round**: the
record carries no round of its own, so the gate reads a count below the marker's round as a
round that never looked, and renders the number STALE rather than as an all-clear.

**Skip this entirely on a PR-flavor doc.** You never edit a PR diff, so no block can be added to
it, and `rows` is not applicable to a scratch by construction. The gate is dormant there too;
recording a ritual `0` every round would be noise that looks like a check.

**`<K>` is a COUNT, not an identity diff, and the gate says so.** Rows are positional (`B<n>`)
and carry line ranges that shift on any edit above them, so a round that REPLACES one block with
another nets zero here and reports no change. Closing that needs a stable per-block identity —
option 2 in #99 — and neither this instruction nor the gate wording may imply it exists.

5. Decide: **converge**, or re-enter `awaiting-secondaries` for another round.

First run `${CLAUDE_PLUGIN_ROOT}/scripts/multi-review-star.sh round-stats "<doc>"`. It prints
Expand Down
28 changes: 26 additions & 2 deletions docs/multi-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,7 +289,10 @@ unrelated to those rows. Its findings still merge normally
way a provider's findings are), so a `defect` still reaches the human gate even from an
otherwise-incomplete turn.

Each round records a durable coverage line alongside the round's quarantine records:
**Round 1 records a durable coverage line** alongside the round's quarantine records — round 1,
not every round: like the symbol-check pass below, this one is derived and dispatched once per
review, because the pairs it compares mostly live in unedited text that a diff-scoped round-N copy
does not carry.

> [crossref-coverage: not applicable]
> [crossref-coverage: <M>/<M> rows verdicted]
Expand Down Expand Up @@ -358,7 +361,8 @@ An exit-1 turn is **reported, never quarantined**, for the same reason as above,
merge normally (`multi-review-star.sh merge --pass "<doc>.symcheck"`, namespaced
`symcheck-rd<N>-<id>`).

Each round records a durable coverage line alongside the round's quarantine records:
**Round 1 records a durable coverage line** alongside the round's quarantine records — round 1,
not every round: this pass is derived and dispatched once per review, not once per round.

> [symcheck-coverage: not applicable]
> [symcheck-coverage: <M>/<M> rows verdicted]
Expand All @@ -368,6 +372,26 @@ Each round records a durable coverage line alongside the round's quarantine reco
`Symbol-check pass: <M>/<M> rows verdicted`, or `Symbol-check pass: <N>/<M> rows verdicted —
INCOMPLETE`.

**Every round records what the pass therefore did not see** (#99), written by the primary at the
END of its own turn — after the edits that address that round's findings, before it touches the
marker. Because the pass runs in round 1 only, its count covers the document as the secondaries
first read it, not the ready-to-paste code the author adds afterwards while fixing findings, which
is the least-reviewed code in the change by construction. Round 1 records one too: its own fixes
are unchecked for exactly the same reason, and a review that converges at round 1 is the commonest
shape of the gap. Each round records:

> [symcheck-added: <K>]

`<K>` is the doc's current `rows` count minus round 1's `<M>` (`0` when that line said `not
applicable`), floored at `0`. The gate renders it under the coverage line — `— round 1 only: <K> more block(s) now than were checked, so at least <K> are
unchecked`, or `— round 1 only: no net change in block count since (a one-for-one swap would not
show here)`, and `— round 1 only: NO RECORD of whether blocks were added since` when no round
recorded one at all. A document where the pass never applied renders none of this.

`<K>` is a **count, not an identity diff**: rows are positional (`B<n>`) and carry line ranges that
shift on any edit above them, so a round that replaces one block with another nets zero. That blind
spot is why the zero wording names it rather than claiming nothing changed.

**Known limits of the introduces set**, in both directions. It harvests the backticked tokens from
`- Create:`/`- Produces:` lines, and a block ends only at a blank line — so a prose-heavy entry
contributes grammar fragments, and an entry that follows the block with no intervening blank is
Expand Down
83 changes: 83 additions & 0 deletions scripts/multi-review-mutation-check.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2139,6 +2139,50 @@ mutations() {
' round N ≥ 2, skip this sub-step entirely** — steps 4, 5, 7 and 8'"'"'s symcheck clauses then have' \
' steps 4, 5, 7 and 8'"'"'s symcheck clauses then have'

# (#99) The pass is ROUND 1 ONLY (the two entries above pin that, deliberately). What was missing
# is any record that later rounds went unchecked: round 1's clean `<M>/<M>` was all the gate saw,
# so a review that added unchecked code rendered identically to one that added none. Three
# separate guards — the instruction to record it, how `<K>` is derived, and the honesty about
# what a COUNT cannot see — each with its own named assertion.
mutate 'command/symcheck-added-recorded' 'commands/multi-review.md' replace \
'no round records blocks added after round 1' 'multi-review-packaging.test.sh' \
' > [symcheck-added: <K>]' \
' > [symcheck-noted]'
mutate 'command/symcheck-added-derivation' 'commands/multi-review.md' replace \
'nothing says how <K> is derived' 'multi-review-packaging.test.sh' \
' the count above minus `<M>`, floored at `0`. An exit 3 from `rows` means the current count is' \
' the count above minus `<M>`. An exit 3 from `rows` means the current count is'

# (fable-rd2-r2) The gate keys STALE off "fewer records than rounds", so the instruction must
# promise exactly one per round — without it a primary recording on its own cadence trips a
# warning it was never told how to avoid.
mutate 'command/symcheck-added-one-per-round' 'commands/multi-review.md' replace \
'nothing states the one-per-round rule' 'multi-review-packaging.test.sh' \
' different facts, and the gate renders them differently. **Exactly one record per round**: the' \
' different facts, and the gate renders them differently. The'

# (fable-rd2-r1) ...and a PR-flavor primary must be told to skip, matching the gate's dormancy.
mutate 'command/symcheck-added-skip-on-pr' 'commands/multi-review.md' replace \
'told to record a count that cannot mean anything' 'multi-review-packaging.test.sh' \
' **Skip this entirely on a PR-flavor doc.** You never edit a PR diff, so no block can be added to' \
' You never edit a PR diff, so no block can be added to'

# (fable-rd1-r1/r4) The paragraph must stay IN THE PRIMARY TURN and say EVERY round. Recorded in
# the fan-out it runs before the body edits, so the count predates this round's own fixes — and
# on the final round those are exactly the blocks nobody checks. The packaging window is scoped
# to the Primary turn section, so weakening this phrase is what the assertion sees.
mutate 'command/symcheck-added-in-primary-turn' 'commands/multi-review.md' replace \
'nothing pins the record to the primary turn' 'multi-review-packaging.test.sh' \
' **Record what the symbol-check pass did not see — EVERY round, and here, not in the fan-out' \
' **Record what the symbol-check pass did not see'

# The blind spot must stay stated. A reader who believes `<K>` detects any change will read a
# zero as "nothing happened", when a one-for-one block swap produces exactly that zero.
mutate 'command/symcheck-added-count-not-identity' 'commands/multi-review.md' replace \
'<K> is presented as if it detected any change' 'multi-review-packaging.test.sh' \
' **`<K>` is a COUNT, not an identity diff, and the gate says so.** Rows are positional (`B<n>`)' \
' **`<K>` has a blind spot.** Rows are positional (`B<n>`)'

mutate 'command/symcheck-not-a-secondary' 'commands/multi-review.md' replace \
'nothing says the symcheck pass is not a secondary' 'multi-review-packaging.test.sh' \
' argument, so **the symcheck pass is not a secondary**: like the crossref pass it is excluded' \
Expand Down Expand Up @@ -2581,6 +2625,45 @@ mutations() {
" [[ \"\$nfoot\" -eq 0 ]] || die \"merge: '\${doc}' carries \${nfoot} already-merged round(s) but '\${doc}.manifest' is missing — refusing to merge onto rounds it cannot verify. Restore the manifest, or rebuild it from the doc's '<!-- star-findings: -->' footers (one 'finding <id>=<hash>' line per entry, every round, in document order, PLUS one 'quarantine <provider>=<hash>' line for each entry in a footer's 'quarantined:' list — verify binds these by hash, so the key needs no round suffix) and re-run\" 1" \
" [[ \"\$nfoot\" -eq 0 ]] || die \"merge: '\${doc}' carries \${nfoot} already-merged round(s) but '\${doc}.manifest' is missing — refusing to merge onto rounds it cannot verify. Restore the manifest, or rebuild it from the doc's '<!-- star-findings: -->' footers (one 'finding <id>=<hash>' line per entry, every round, in document order) and re-run\" 1"

# (#99) ...and the gate must RENDER it. FIVE outcomes, five entries: a positive count, a zero that
# does not overclaim, a NO RECORD when nothing was written, a STALE when some round forgot, and
# silence on a PR scratch. This comment has now been wrong about the count twice — "three
# branches" (fable-rd1-r3 found the zero branch uncovered) and then four (fable-rd2-r1/r2 added
# the flavor and staleness branches). Count the `echo`s before trusting it again.
mutate 'star/gate-symcheck-added-rendered' 'scripts/multi-review-star.sh' replace \
'round-1-only gap is silent' 'multi-review-star.test.sh' \
' echo " — round 1 only: ${sadd} more block(s) now than were checked, so at least ${sadd} are unchecked"' \
' :'
mutate 'star/gate-symcheck-added-no-record' 'scripts/multi-review-star.sh' replace \
'an unrecorded later round is silent' 'multi-review-star.test.sh' \
' echo " — round 1 only: NO RECORD of whether blocks were added since"' \
' :'
# (fable-rd1-r3) The ZERO branch needed its own entry. The rendering has four outcomes, not the
# three the first cut of this comment claimed, and the one it left uncovered is the anti-overclaim
# wording the whole design leans on: a COUNT cannot see a one-for-one swap, so a bare "no change"
# here would assert more than the number knows.
mutate 'star/gate-symcheck-added-zero' 'scripts/multi-review-star.sh' replace \
'zero added-count rendered nothing or overclaimed' 'multi-review-star.test.sh' \
' echo " — round 1 only: no net change in block count since (a one-for-one swap would not show here)"' \
' :'

# (fable-rd2-r1) A PR scratch must stay silent, and the COVERAGE ENCLOSURE DOES NOT DO THAT — the
# first cut of this entry said it did, which was simply false: `rows` exits 3 on a scratch, the
# round records `not applicable`, and a non-empty `sx` reaches the block. Only the flavor branch
# keeps it out. The primary never edits a PR diff, so the warning could never be true there.
mutate 'star/gate-symcheck-added-dormant-on-pr' 'scripts/multi-review-star.sh' replace \
'was warned about unchecked blocks it cannot have' 'multi-review-star.test.sh' \
' if [[ "$(_doc_flavor "$doc")" != pr ]]; then' \
' if true; then'

# (fable-rd2-r2) A count that is not current is worse than none — it reads as an all-clear. The
# record carries no round, and the reader is `tail -1`, so without this a round that forgot to
# record renders an earlier round's number as this round's fact: #99's own gap, one round down.
mutate 'star/gate-symcheck-added-stale' 'scripts/multi-review-star.sh' replace \
'rendered as current on a round-3 review' 'multi-review-star.test.sh' \
' elif [[ "$srd" =~ ^[0-9]+$ ]] && (( sn < srd )); then' \
' elif false; then'

# gate-summary renders the claim BEFORE the human approves the publish. Nothing can mechanically
# verify a prose finding was fixed, so the gate is the only thing standing behind it — a silent
# gate summary means the human approves a claim they were never shown.
Expand Down
Loading
Loading