Committed Himes fixture (himes-parent) is stale and doesn't match the paper's case study (57bd1b53), which isn't in any repo

[andrewsu]

Summary

The only executed Himes package committed to the repo (testdata/replay/himes-parent/) is stale and does not correspond to the case study reported in the paper (session 57bd1b53), and the paper's session is not present anywhere in the repo. A reader who clones the repo to inspect "the Himes run" finds numbers that contradict every headline figure in the paper, with nothing indicating the two are different sessions. Because the claim set is non-deterministic (see #5 / the narrative-generation boundary), the paper's run cannot be regenerated — so unless the exact package is deposited, the paper's central numbers are unverifiable.

Evidence

The committed fixture is known-stale (by our own test code):

• crates/core/tests/provenance/fresh_executed_crate.rs: "The committed testdata/replay/himes-parent executed fixture is stale."
• crates/core/tests/provenance/replay_himes.rs: "testdata/replay/himes-parent/ is a trimmed copy of a real completed bulk RNA-seq package fixture." Only data_acquisition + differential_expression outputs survive.

It contradicts the paper on every headline number:

	committed fixture (himes-parent)	paper case study (57bd1b53)
session	9bffd538	57bd1b53
date	2026-06-23	2026-07-11
tasks	19 (count matrix supplied, upstream pruned)	22
candidate claims	122 (90 verified / 32 unverifiable)	72 (25 verified / 47 unverifiable)
LFC shrinkage	normal-prior	apeglm
in repo?	yes	no — 0 files mention 57bd1b53

(There is a third Himes artifact adding to the confusion: testdata/emitted-packages/09-himes-dex-airway/ is a 33-task from-FASTQ, compile-only plan with 0 claims — neither the fixture nor the paper run.)

Why it matters

• A reviewer inspecting the repo's Himes run gets 122 claims / 19 tasks / normal-prior, none of which matches the paper's 72 / 22 / apeglm / 4,030 sig genes. Nothing labels the fixture as a different session, so this reads as a discrepancy or error even though it's legitimate.
• The paper's Data/Code Availability implies the case study is inspectable; the actual published package is currently unreachable (57bd1b53 is in no repo; the Zenodo deposit is "in preparation").
• The claim set is a non-deterministic, one-time product of the agentic run — it is not regenerable. So depositing the exact 57bd1b53 package is the only way the paper's 72-claim result (and the reverse-traversal figure, self-audit, etc.) can ever be checked by a third party.

Recommendation (unify on one canonical session)

A separate, minimal replay fixture is legitimate — the full 595-entity case-study package (with 15 non-regenerable network/literature outputs) should not be committed to testdata/. But the three divergent artifacts should be collapsed into one coherent story:

1. Deposit 57bd1b53 to Zenodo and cite the DOI in the paper. Highest priority: it is otherwise unverifiable.
2. Replace the stale himes-parent with a trimmed slice of 57bd1b53 so the repo fixture's session id / DE table / significant-gene count reconcile with the paper, and the replay test becomes a re-verification of the actual published package.
3. Add a disambiguation note in testdata/replay/ regardless: which session the fixture is, that it is a replay-machinery test fixture, and (if not identical to the paper run) a pointer to the published Zenodo DOI.
4. Soften "reproducible" language for the claim results — the compute path is reproducible; the claim set is an inspectable recorded artifact, not a reproducible one (relates to literature_review_requested is hardcoded true at emit — recorded intent field doesn't reflect SME intent #5).

Related

Companion to #5 (both concern provenance/reproducibility fidelity of the emitted record).

---

_{Filed from a source review against main (8cd0f34).}