Skip to content

[Dream Cycle 2026-10-11] ledger-signals: duplicateDirections counts re-appended rows as distinct nights (false positive) + witness,verify scan #166

Open

Description

@ruvnet

Rotation

Slot 1 (DAYINT % 5 == 1) → DEEP=ledger-signals, SCAN=witness,verify. No bonus moduli (DAYINT % 25, DAYINT % 75 both non-zero).

Ledger Check

Last row on main: 2026-09-30 (PR #143). daysSinceLastRow=11, ledgerStale=true. Ground-truth GitHub check (search_pull_requests, not the unreliable list_pull_requests.merged field): 36 dream/* PRs merged all-time, most recent #143 on 2026-09-30; 14 dream/* PRs open/unmerged since then (#98, #101, #137, #139, #141, #145, #148, #150, #152, #154, #156, #158, #161, #165) — reviewBacklogSize=14, zeroMergeStreak=false (computed with real --merged ground truth over the last 14 ledger rows).

Deep Dive (ledger-signals)

learningSignals()'s duplicateDirections signal exists to detect "a finding repeated in >= 3 PRIOR NIGHTS" (STEP 1.1) so the agent rotates off an over-mined direction. It counts raw row occurrences of a normalized finding key with no per-date dedup. The real docs/dream-cycle/LEDGER.md violates the 1-row-per-night assumption this relies on (a known, already-flagged defect — PR #129/2026-09-21 added distinctDatesInWindow as a diagnostic for this exact row/night conflation but explicitly left duplicateDirections itself as named future work). Concretely: the 2026-08-28 "harden anchored replay JSON evidence…" finding was appended 3 times on the same date (PR-number churn), and duplicateDirections flags it as a 3-night repeat — one night, not three. An existing unit test even locked the bug in (3 rows via the row() helper, no date override, so all 3 silently shared one default date).

Hypothesis

Given the real committed ledger (one night re-appended 3x under the identical finding text), when duplicateDirections counts by raw row occurrence instead of distinct dates, then a single re-appended night is mis-counted as 3 independent nights and falsely crosses the rotate-away threshold, subject to: a genuine repeat across 3 actually distinct dates must still be flagged, and pendingFindings (open-PR findings, no row date) must keep contributing one occurrence each, unchanged.

Evaluation Receipt

Evaluator: npm test (vitest run && npm run test:governance), parent then candidate on identical source.

  • Parent (5707c825b2c065de44abed02594e1abf5eab95ea): vitest 769/769, governance 159/159 → 928/928.
  • Candidate: vitest 770/770 (+1), governance 159/159 (unchanged) → 929/929, 0 regressions. typecheck/lint clean.
  • Live: node packages/cli/dist/bin.js ledger signals --path docs/dream-cycle/LEDGER.md on the real ledger — duplicateDirections contains the 2026-08-28 false positive pre-fix, empty of it post-fix.

Darwin Results

Not run — optional, and this is a small deterministic library fix with no evolutionary-search use case; evaluatorEntrypoints.darwin is still an unpinned npx supply-chain risk (standing warning, tracked separately by #157/#158).

Evidence

MEASUREMENT: parent/candidate test counts above, both read directly from npm test output. OBSERVATION: real ledger rows 29–31 (2026-08-28) share identical finding text, same date, three times. DECISION: dedupe duplicateDirections' row-counting by (directionKey, date).

Reward-Hack Check

Independent subagent critic (no authorship stake) read the diff and both source files, independently reproduced the real duplicate ledger rows, independently re-ran the test file (46/46), and checked pendingFindings/blank-date edge cases. Verdict: clean fix, no threshold touched, no cache, no narrowing to just pass the new test; pendingFindings path left byte-identical and re-tested as such.

Security Review

No security-sensitive surface: no prompt injection, credential, filesystem/network-scope, or agent-authority change — pure in-memory signal bookkeeping.

Scan Findings (witness, verify)

Competitors

Sakana AI Scientist (B), OpenHands (A), DSPy/GEPA (A), SWE-agent (A), AutoGPT lineage (C) — none maintain a hand-editable, multi-writer markdown ledger with a "rows ≈ nights" assumption (this repo's own ADR-0001 design), so this is a repo-specific correction with no external precedent to borrow. Full writeup in the committed report.

Gist

GIST=LOCAL — no gh/gist-creation capability available this session (gh auth status reports an invalid GH_TOKEN); report committed directly as the durable artifact: docs/dream-cycle/2026-10-11-ledger-signals-report.md.

Witness

report_sha256 : d2a9e19a68680a40e8fa3bdc90718d2b5088e6bcf22910ba25cd682d52cbd8dc
session_commit: 5707c825b2c065de44abed02594e1abf5eab95ea
witness       : 54824cd4528bcc15932c98bd5d97dd2222271f571a1f7fd0ce32a29ddf790401

Self-verifiable: node packages/cli/dist/bin.js witness verify-report docs/dream-cycle/2026-10-11-ledger-signals-report.md --commit 5707c825b2c065de44abed02594e1abf5eab95ea

Recommendation

evaluated: accepted. Human review requested on the PR. Small, additive, fully covered by tests; no production behavior change beyond the one signal's row-counting discipline.

Activity

  1. added 2 commits that reference this issue on Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions