Train and score the §4 ablation on InterPro 85.0 (t0) domain architectures

RELATIVE_INFERENCE_REPORT.md

@@ -13,8 +13,8 @@ human-only run of that reimplementation:
 
 - **Training annotations:** human UniProtKB-GOA, archived release 205
   (April 2021), manual (non-IEA) evidence codes — 18,735 annotated proteins.
-- **Domains:** InterPro entries (InterPro `current_release` snapshot of
-  2026-07-22), plus contiguous "supra-domain" combinations of up to three
+- **Domains:** InterPro entries (InterPro `current_release` as downloaded
+  2026-07-07), plus contiguous "supra-domain" combinations of up to three
   entries (as in the original method, which used SCOP superfamilies).
 - **Ontology:** GO, `go-basic.obo` release 2026-06-15, `is_a`/`part_of`
   edges. Input annotations are true-path propagated before testing, as the
@@ -385,15 +385,17 @@ the same cells either way.
 **Robustness across the design axes.** The same four configurations were
 rerun with the domain key switched to SCOP superfamilies (`ssf`, the
 published dcGO's domain universe) and with IEA annotations included in
-training — four cells in all. The Base → Base + relative contrast, paired
-bootstrap per cell:
+training — four cells in all — and, in the last row, on 2021 domain
+architectures. The Base → Base + relative contrast, paired bootstrap per
+cell:
 
 | cell | cohort BP / MF / CC | F_max: relative worse | AUPRC: relative worse |
 |---|---|---|---|
 | InterPro, manual | 298 / 406 / 436 | 8/9 (5 sig; 0 sig better) | 8/9 (5 sig; 1 sig better) |
 | InterPro, + IEA | 187 / 242 / 161 | 8/9 (3 sig; 0 sig better) | 8/9 (5 sig; 1 sig better) |
 | SCOP SSF, manual | 140 / 221 / 195 | 7/9 (6 sig; 2 sig better) | 7/9 (6 sig; 1 sig better) |
 | SCOP SSF, + IEA | 72 / 104 / 51 | 8/9 (2 sig; 1 sig better) | 8/9 (6 sig; 1 sig better) |
+| InterPro 85.0 (t0), manual | 298 / 406 / 436 | 7/9 (5 sig; 0 sig better) | 8/9 (5 sig; 1 sig better) |
 
 The direction is the same in every cell: the relative test is worse in 7–8
 of 9 aspect × IC cells on both metrics, with at most two significant gains.
@@ -407,6 +409,12 @@ is always built from exactly the evidence training saw. The SSF cells keep
 only proteins carrying SSF domains. So absolute numbers are not comparable
 across cells; the within-cell contrasts are.)
 
+On 2021 architectures (last row: the InterPro/manual cell retrained on
+InterPro 85.0, released 2021-04-08, and scored on the same cohort), Base is
+slightly lower, by up to 0.022 F_max in MF. The same cells are significant,
+in the same direction, so domain-side look-ahead does not explain the result
+either (`VALIDATION_PLAN.md` §4 "t0 InterPro architectures").
+
 ### 4.3 Sensitivity / PPV decomposition at the operating point
 
 *(Recall = sensitivity. Precision = positive predictive value, PPV — the
@@ -591,17 +599,24 @@ test with its measured cost rather than as a default.
   all of them clean their input. Inputs are unchanged: training GAF
   sha256 `69ae7d90…` (= archived GOA release 205),
   `data/interim/protein2ipr_human.dat.gz` sha256 `a932a515…` (InterPro
-  `current_release` snapshot of 2026-07-22), `go-basic.obo` sha256
+  `current_release` as downloaded 2026-07-07), `go-basic.obo` sha256
   `c72fc198…` (release 2026-06-15). Design and prose: `VALIDATION_PLAN.md`
   §4.
 - **Regeneration check**: the committed manifests come from a clean
   checkout of the branch (`78494f2`; `git.dirty: false` in
   every manifest), and an independent earlier run of the same 36 pipeline
   configurations from a working tree reproduced every association table
-  **byte-for-byte** (96/96 output TSVs).
+  **byte-for-byte** (96/96 output TSVs). Those runs read the 2021 GAF through
+  a link at the default GAF path; to regenerate, run each manifest's recorded
+  command with `--gaf data/raw/goa_archive/goa_human.gaf.205.gz` added
+  (`VALIDATION_PLAN.md` §4, "Regenerating a cell").
 - **Robustness cells** (§4.2): the same nine-rung ladder rerun with
   `--domain-key ssf` and/or `--evidence-filter all`; eval tables and
-  per-run manifests in `validation/ablation_cells/{ipr_all,ssf_manual,ssf_all}/`.
+  per-run manifests in `validation/ablation_cells/{ipr_all,ssf_manual,ssf_all}/`;
+  the InterPro 85.0 (t0) rerun and its selection control in
+  `validation/ablation_cells/{ipr_manual_t0interpro,ipr_manual_selctrl}/`,
+  paired against the committed cell in
+  `validation/ablation_cells/t0_interpro_comparison.tsv`.
 - **Scoring conventions** (§4.0, §4.2): `validation/check_metric_conventions.py`
   re-scores the cell with F_max only at τ ≥ 0.001 and AUPRC without its
   point at the origin, on the committed bootstrap resamples; output

REPRODUCIBILITY.md

@@ -52,6 +52,17 @@ Hashing is intentionally part of the run. For large inputs it adds sequential
 I/O, but prevents a mutable filename from silently referring to different data.
 The species-specific InterPro file is a derived input: its own hash identifies
 what was analyzed, while `derived_from` records the upstream mapping source.
+Inputs passed with `--gaf`/`--interpro` (archived snapshots for the temporal
+benchmark) are hashed and header-read the same way but carry no
+`source_url`/`derived_from`: the species' current-release URL would name
+different bytes. A subset's own `.provenance.json` sidecar names the archive it
+was cut from, the GAFs (path and SHA-256) and the evidence filter that
+selected it;
+`scripts/run_t0_interpro_ablation.py` copies it into the evaluation's
+`manifests/` so it is committed with the results. The two sidecars committed
+under `validation/ablation_cells/ipr_manual_{t0interpro,selctrl}/` predate
+the `evidence_filter` and `selection_sha256` fields; their extraction logs
+record `Evidence codes: ALL` (`VALIDATION_PLAN.md` §4).
 
 The inputs recorded are exactly those the selected ontology declares in
 `src/ontology_registry.py` (`needs`, plus `hierarchy_needs` when

RESULTS.md

@@ -202,25 +202,20 @@ Assumes the human InterPro subset is already extracted
 # 1. Fetch the 2021 t0 GOA snapshot
 uv run python scripts/download_data.py --goa-archive 205
 
-# 2. Stage t0 inputs under the --species name run_dcgo_human.py expects.
-#    Domains are fixed in time (only GOA moves), so symlink the current subset.
-ln -sf "$PWD/data/raw/goa_archive/goa_human.gaf.205.gz" \
-       data/raw/goa_annotations/goa_human_t0_2021.gaf.gz
-ln -sf "$PWD/data/interim/protein2ipr_human.dat.gz" \
-       data/interim/protein2ipr_human_t0_2021.dat.gz
-
-# 3. Train domain→GO associations on t0
-uv run python run_dcgo_human.py --species human_t0_2021 --num-cores 32 \
-    --output-dir results_t0_2021
-
-# 4. Protein-centric temporal benchmark (t1 = current GOA)
+# 2. Train domain→GO associations on t0. Only GOA moves: the domains are the
+#    current subset (the --interpro default). VALIDATION_PLAN §4 has the rerun
+#    on InterPro 85.0 (t0) architectures.
+uv run python run_dcgo_human.py --gaf data/raw/goa_archive/goa_human.gaf.205.gz \
+    --num-cores 32 --output-dir results_t0_2021
+
+# 3. Protein-centric temporal benchmark (t1 = current GOA)
 uv run python validation/temporal_benchmark.py \
     --t0-gaf data/raw/goa_archive/goa_human.gaf.205.gz \
     --t1-gaf data/raw/goa_annotations/goa_human.gaf.gz \
     --predictions results_t0_2021/domain_go_associations_significant.tsv \
     --min-ic 0 --min-ic 2 --min-ic 4 --min-ic 6
 
-# 5. Domain-centric eval — build the relative-filtered set, then compare both
+# 4. Domain-centric eval — build the relative-filtered set, then compare both
 uv run python validation/apply_relative_inference.py \
     --predictions results_t0_2021/domain_go_associations_significant.tsv \
     --t0-gaf data/raw/goa_archive/goa_human.gaf.205.gz \

TODO.md

@@ -25,6 +25,17 @@ implementation.
   archived t0 InterPro/protein2ipr snapshot, or explicitly limit every claim to
   an annotation-temporal benchmark. This also closes the 31.9% universe loss in
   the all-species held-out arm.
+  **§4 ablation done on t0 architectures (2026-10-07).** The `ipr_manual`
+  cell was retrained and rescored on InterPro 85.0 (2021-04-08), with a
+  selection-control arm, and paired on the committed cohort
+  (`VALIDATION_PLAN.md` §4 "t0 InterPro architectures"). Base drops by up to
+  0.022 F_max (MF). No conclusion changes. Still open:
+  - the §2 headline benchmark, the multi-ontology breadth test, the
+    surprise-score held-out test, and the all-species and mouse arms still read
+    current `protein2ipr` (the code path exists: `--interpro`,
+    `extract_human_interpro.py --source`);
+  - GO-vocabulary look-ahead: seven of nine rungs train with `go-basic.obo`
+    2026-06-15 (propagation, parental background, retired-id remap).
 - [ ] **Correct phylogenetic non-independence (gene / UniRef50 collapse).**
   Protein-level pooling inflates all-species support by about 2.44× and half of
   associations rest on at most three UniRef50 clusters; the model-organism and