Skip to content

ledger-signals: blockedEvalStreak ignores its own "majority of window" doc comment (issue #153) - #154

Draft
ruvnet wants to merge 6 commits into
mainfrom
dream/2026-10-06-ledger-signals
Draft

ruvnet wants to merge 6 commits into
mainfrom
dream/2026-10-06-ledger-signals

Conversation

@ruvnet

@ruvnet ruvnet commented Oct 6, 2026

Copy link
Copy Markdown
Owner

Nightly Dream Cycle, 2026-10-06. DEEP=ledger-signals, SCAN=witness,verify. Full report: docs/dream-cycle/2026-10-06-ledger-signals-report.md. Issue: #153.

Hypothesis (frozen before implementation)

Given a 14-row window where exactly the last 3 rows have evaluated: 'blocked' and the rest do not (blocked rows a strict minority), the pre-fix implementation returns blockedEvalStreak: true (false positive, since 3/14 is not "majority of the window blocked" as the field's own code comment requires alongside last-3-consecutive); the fix adds the missing majority conjunct so the same input returns false, while an all-blocked window (existing passing test) and a genuine-majority window continue to return true.

Candidate

packages/ledger/src/index.ts (+11/−1): learningSignals()'s blockedEvalStreak gains a blockedCount tally and a blockedCount * 2 > recent.length (strict-majority) conjunct, in addition to the existing last-3-consecutive-blocked check — implementing exactly what the line's own pre-existing doc comment already promised. packages/ledger/src/index.test.ts (+27): 3 new tests (minority-tail false positive fixed; genuine majority still true; exact 50/50 split stays false). One conceptual change.

Evaluation Receipt

Real evaluator: npm test (vitest run && npm run test:governance).

Baseline (main@23a5770) Candidate
vitest 769 772 (+3 new, 0 regressions)
governance (node --test scripts/*.test.mjs) 150/150 150/150 unchanged
npm run lint clean clean
npm run typecheck clean clean
npm run build (8 packages) clean clean

Governance is 150/150 clean on both sides this session — this session's checkout started shallow (3/150 failing on scripts/benchmark-ticket-codec.mjs's pinned BASELINE_COMMIT git-show oracle, absent from the shallow clone) and was unshallowed (git fetch --unshallow) before measuring either side. That exact shallow-clone defect is already an open, independent finding this rotation did not select tonight (PR #150, security-adversarial, 2026-10-03) — not duplicated here.

Live pre/post-fix repro against the built packages/ledger/dist/index.js:

  • Pre-fix: 11 yes + 3 blocked (tail) in a 14-row window → blockedEvalStreak: true (false positive, 3/14 ≈ 21%).
  • Post-fix: identical input → false.
  • Post-fix, genuine majority: 6 yes + 8 blocked (tail) in a 14-row window → true (unchanged — majority case was never broken).
  • Real committed LEDGER.md: blockedEvalStreak: false identically before and after — this bug is latent on main (never yet triggered there); the fix forecloses the next time OPENROUTER_API_KEY is briefly absent for 3 consecutive nights.

Baseline

Parent commit 23a577082aad39750051dc7ef0ec9d0d72900adf (main, HEAD at session start), rebuilt clean (npm ci && npm run build), 769/769 vitest + 150/150 governance.

Darwin Lineage

DARWIN=not-applicable — single boolean-expression fix, no evolvable population (same judgment as PRs #116/#127/#132/#137/#145 on this surface). Entrypoint probed live tonight: npx --yes @metaharness/darwin --version responds with its usage banner (reachable, not blocked).

Evidence

OBSERVATION (the field's own doc comment names two conjuncts; the implementation computed only one) → MEASUREMENT (live pre-fix repro: 3/14 minority-blocked tail reads true) → OBSERVATION (Resilience4j/Hystrix circuit-breaker docs confirm count-alone vs. rate-over-window is a known false-positive class in comparable "sustained failure" signal designs) → MEASUREMENT (769→772 vitest, 0 regressions; governance 150/150 unchanged) → MEASUREMENT (live post-fix repro: minority false, majority still true, real ledger unchanged) → INFERENCE (independent critic subagent, separate context, own build/stash cycle) → DECISION (ACCEPT).

Reward-Hack Check

Independent adversarial-critic subagent (separate context, own commands, own git stash/rebuild cycle): CLEAR.

  • Confirmed the diff touches only packages/ledger/src/index.ts (+11/−1) and its test file (+27/−0, pure addition) — no gold data, evaluator, or CI config touched.
  • Independently rebuilt and ran the suite before/after via its own git stash: 769→772 (exactly +3, matching the 3 new tests; no existing test deleted, weakened, or had an assertion loosened).
  • Wrote its own throwaway probe (independent of this candidate's tests) and tried 6 edge cases by hand: exact-half (50/50) correctly false; 8/14 majority correctly true; majority concentrated at the window head with a healthy tail correctly false (last-3-contiguous requirement still enforced); below-minimum window (n=2) correctly false; degenerate n=3 all-blocked correctly true; non-default window option correctly rescales the majority base rather than hardcoding 14. No counter-example found.
  • Repo-wide grep for blockedEvalStreak: the only other reference (packages/cli/src/tui.ts:206) merely prints a warning string when true — no gating/CI/evaluator logic anywhere depends on the old (broken) last-3-only semantics.
  • Residual non-blocking note from the critic: no test exercises a non-default window option through learningSignals itself end-to-end (only through its own ad hoc probe) — a minor coverage gap, not a defect, left for a future night.

No gold data, threshold, or benchmark corpus touched; no undocumented cache; no hidden cost (same asymptotic cost — one extra O(n) filter over an already-sliced window).

Security Review

No credential, network, or trust-boundary surface added or changed. Pure local arithmetic/boolean logic over already-parsed in-memory rows. No LLM calls in this path (OPENROUTER_API_KEY present this session, correctly unused).

Regression Analysis

0 pre-existing tests modified or removed (confirmed independently by the critic via raw diff). 3 new vitest tests added. Lint/typecheck/build clean across all 8 packages — no wasm/NAPI degradation to record tonight.

ADR

None — a correctness bug fix aligning an existing field's implementation with its own pre-existing doc comment, not a new architectural decision.

Gist

GIST=LOCAL — no gh CLI (GH_TOKEN invalid) and no gist-creation MCP tool this session; GitHub MCP access itself works fine (issue/PR tools succeed), so FALLBACK=false. Full report committed at docs/dream-cycle/2026-10-06-ledger-signals-report.md.

Issue

#153

Witness

report_sha256 : 274687e0e453001e7c9fb554a4da77d258a9c3608005f43fb43630618e631305
session_commit: 23a577082aad39750051dc7ef0ec9d0d72900adf
witness       : 907bbf1fb1279c979dca95414c95a12e19e58e4dd6c05070f81f33483ecfdc44

Reproduce: dream-machine witness verify-report docs/dream-cycle/2026-10-06-ledger-signals-report.md --commit 23a577082aad39750051dc7ef0ec9d0d72900adf

Merge Policy

Human review required. This session never self-merges and never autonomously promotes candidate state. dream.config.json's autoMerge: false; no automerge-safe label applied.


🤖 Generated with Claude Code

https://claude.ai/code/session_01SqtjzJmtG789iUrkQU8bfz


Generated by Claude Code

claude added 3 commits October 6, 2026 09:28
…" doc comment (issue #153)

learningSignals()'s blockedEvalStreak comment says "majority of the window
blocked, and last 3 all blocked" but the implementation only ever checked
the trailing-3 condition. 3 blocked nights buried in an otherwise-healthy
14-night window (a clear minority) false-positived the signal, which this
same nightly pipeline reads for its own STEP 1.1 planning. Added the
missing blockedCount*2 > recent.length strict-majority conjunct.

npm test 769->772 (+3 new, 0 regressions); governance 150/150 unchanged;
independent adversarial-critic subagent verdict: CLEAR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqtjzJmtG789iUrkQU8bfz
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqtjzJmtG789iUrkQU8bfz
… main, ported)

dependency-audit (npm audit --audit-level=high, dev-inclusive graph) started
failing on main itself today (reproduced on a fresh main clone, unrelated to
this PR's diff): a new high-severity advisory for source-map-js
(GHSA-68fv-2mgg-jv7q, event-loop DoS via indexed source-map section offsets)
was published to the npm advisory database after main's last green CI run
(2026-09-30). source-map-js is a transitive devDependency (vite->postcss,
@vitest/coverage-v8->magicast) with no production reachability.

npm audit fix resolves it with a non-breaking, lockfile-only bump
(fixAvailable: true, no --force needed). Ported here since this PR is
otherwise ready and waiting on this unrelated base-branch failure; it will
no-op once main also carries it.

npm test 772/772 vitest + 150/150 governance unchanged; lint/typecheck clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqtjzJmtG789iUrkQU8bfz

ruvnet commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

dependency-audit (CI) failed on commits 21a2cf3/4c99573 — not this PR's defect. Root-caused and fixed:

Not this PR's: dependency-audit runs npm audit --audit-level=high against the full dev-inclusive dependency graph, unrelated to anything this diff touches. Reproduced identically against a fresh clone of main itself today: a new high-severity advisory, source-map-js GHSA-68fv-2mgg-jv7q (event-loop DoS via indexed source-map section offsets), was published to the npm advisory database sometime after main's last green CI run (2026-09-30, commit 23a5770). source-map-js is a transitive devDependency only (vite→postcss, @vitest/coverage-v8→magicast), no production reachability.

Fix ported, pushed: npm audit fix resolves it with a non-breaking, lockfile-only bump (fixAvailable: true, no --force/major bump needed) — commit 330df89. npm audit --audit-level=high → found 0 vulnerabilities locally; full suite re-validated (772/772 vitest, 150/150 governance, lint/typecheck clean). This will no-op harmlessly once main also picks up the same fix.

No fix PR for this advisory existed yet (checked via search_pull_requests), so this isn't a duplicate — tracking it here rather than opening a separate PR since it's a one-line lockfile bump riding this PR's next push.


🤖 Generated with Claude Code

https://claude.ai/code/session_01SqtjzJmtG789iUrkQU8bfz


Generated by Claude Code


Generated by Claude Code

ruvnet commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

CI is green on the current head (330df89) — all 9 checks passing, no merge conflict, no open review threads. Waiting on human review.


🤖 Generated with Claude Code

https://claude.ai/code/session_01SqtjzJmtG789iUrkQU8bfz


Generated by Claude Code


Generated by Claude Code

… consolidation

# Conflicts:
#	docs/dream-cycle/LEDGER.md
Merge main (3fc3222, unchanged since 2026-10-19) into this branch;
re-verify the blockedEvalStreak majority-conjunct fix on the new head
(889 vitest +3, 195 governance unchanged, 0 regressions); fresh critic
pass finds no reward-hacking across 7 adversarial probes. dependency-audit
stays red on a pre-existing vite/launch-editor advisory (issue #197, not
this PR's) so this stays draft per STEP 1.5. One ledger row appended.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013xi2Ev8ACA38fPxKmWWJyK

ruvnet commented Oct 25, 2026

Copy link
Copy Markdown
Owner Author

Dream Cycle consolidation — 2026-10-25

MODE=CONSOLIDATE (STEP 1.5, ADR-0011): 20 open dream/* candidate PRs tonight, well over implementationTrack.reviewBacklogLimit=6. The three nearer-the-front eligible drafts — #145, #148, #150 — each already carry a dream-consolidate marker for today's exact default-branch SHA (3fc3222d2e5566a203331d3686c4d2eaf06713a8) from the three preceding nights; main has not moved since 2026-10-19 (#196), so there is no new SHA for them to re-consolidate against. This PR was the next-oldest still-eligible draft (still draft, ACCEPT in its own Evaluation Receipt, not a routine-self-review PR) carrying no prior consolidate marker.

Action. Merged main (3fc3222d2e5566a203331d3686c4d2eaf06713a8) into this branch with an ordinary merge commit (9110482, never rebase, never force-push). One textual conflict in docs/dream-cycle/LEDGER.md (both sides had appended rows since this branch's 2026-10-06 base) — resolved by chronological concatenation, no row lost either side. packages/ledger/src/index.ts auto-merged clean against substantial intervening main history on the same file (the Dream Cycle v2 engine landing in #196, plus PR #167's duplicateDirections de-dup) — confirmed by direct inspection that blockedCount/blockedEvalStreak's majority conjunct is present, unaltered, in the merged file.

Re-evaluation on the new head. Real evaluator (npm test): vitest 886→889 (+3, this candidate's own tests, 0 regressions vs. main's own 886), governance 195/195 unchanged. npm run typecheck/npm run lint clean; npm run build clean (8 packages, no wasm/NAPI degradation). node packages/cli/dist/bin.js ledger verify --path docs/dream-cycle/LEDGER.md --since-row 38 --legacy-digest 23b02bfa98b56b374fadfb1b05df25e589dcdeab26143ae6183083d21779426a: OK. Live repro re-run against the rebuilt packages/ledger/dist/index.js: minority tail (3/14) → false; genuine majority (8/14) → true; all-blocked (14/14) → true; exact 50/50 (7/14) → false — all unchanged from the original round.

Fresh critic + reward-hack pass (this round). Disclosed limitation: nested-agent spawning was denied at depth 1 this session, so no separate subagent context was available — this pass was run directly by the consolidating session rather than a fully isolated subagent. Diffed the candidate's two files from the PR's original base (23a5770) to the new head: byte-identical to the originally reviewed diff — no competing change landed on blockedEvalStreak itself since baseline. Ran 7 of my own adversarial probes beyond the shipped 3 tests directly against the rebuilt dist: empty window (false, no crash), n=1 blocked (false), minimum n=3 all-blocked (true), custom window=3/5/6 options correctly rescaling the majority threshold (true/true/false), and an alternating pattern where majority is blocked but the tail isn't contiguous (false, last-3-consecutive still enforced). No counter-example found; no gold data, threshold, or test weakened. CLEAR.

CI on the pushed head. Green: guard, CodeQL, test (both Node versions), no-optional-deps, software-evidence, dependency-security. Red: dependency-audit (npm audit --audit-level=high) — reproduced locally against this exact head's lockfile; git diff main -- package-lock.json from this branch is empty, so the lockfile is byte-identical to main's. Same pre-existing, already-tracked vite 8.3.0–8.3.2 → launch-editor dev-toolchain advisory as issue #197 (opened 2026-10-21 by this same consolidation pattern on PR #145, still open, no fix PR exists yet to port — a semver-range-breaking vite/vitest major bump is out of scope for this single-boolean-expression fix). Not re-filing a duplicate issue; not fixing it from this PR.

Decision. ACCEPT still holds and the fresh critic is clear, but not every required check on the new head is green (dependency-audit, for the reason above) — per STEP 1.5, this PR stays draft. Report refreshed at docs/dream-cycle/2026-10-06-ledger-signals-report.md with a new Evaluation Receipt addendum and witness (1ed38f45…, self-verified via dream-machine witness verify-report). Ledger row appended: docs/dream-cycle/LEDGER.md (2026-10-25).

No merge, no close, no force-push, no self-promotion.


Generated by Claude Code


Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants