gains: la verdad de una ventana, con su estrato y su cierre

CLAUDE.md

@@ -136,7 +136,7 @@ The frontend (`apps/web/`) is a Next.js 16 app with Tailwind v4. API URL is conf
 - `export_evaluation_targets` (`ExportEvaluationTargetsOperation`): writes an `EvaluationSet`'s delta proteins as a FASTA and publishes it to the artifact store, so the same bytes are both the LAFA `--query_file` and the `QuerySet` the internal grid dispatches against. Exists to close an asymmetry: `apps/method_runtime/protea_predict.py` REQUIRES `--query_file`, while `PredictGOTermsPayload` leaves `query_set_id` and `query_accessions` optional and silently selects every protein with an embedding when neither is given. That is a comparability defect before it is a cost one, since a grid scoring one population and a submission scoring another produce numbers that are not about the same thing. Targets are the union of the `nk` / `lk` / `pk` maps, the proteins that GAINED annotation; `removed` holds proteins that only lost annotation, which the evaluation reports and never scores, and requesting it as a category raises. Accessions are sorted and sequences wrapped at a fixed width so the same evaluation set always produces the same bytes, and the returned sha256 is what lets both consumers prove they scored one population rather than assert it. Headers are bare accessions, which round-trip through LAFA's parser unchanged. Delta proteins with no stored sequence are emitted as a warning rather than dropped silently. Runs on `protea.jobs`.
 
 - `declare_query_set_provenance` (`DeclareQuerySetProvenanceOperation`): recovers a hand-registered `QuerySet`'s derivation by **verifying** it rather than reading it out of the description. Four sets were registered through the FASTA upload endpoint by a person, with the window as an id prefix, the job as a word and the sha as a string nothing compared against; migration `c5e8d1a30b72` gave the table the columns and deliberately declared nothing, because recovering a derivation is a judgement that belongs in a job with its events. For each named set it finds every SUCCEEDED `export_evaluation_targets` whose published target count equals the entry count, fetches each one's FASTA through `fetch_and_verify` with the key **derived** from what the job declared (`targets_key_for`, the one spelling, so no later reader has to split a stored uri the way the ground truth's second door did), and writes the window, sha256, job and revision only when the accession set in those bytes equals the set's entries exactly. The uri it stores is the one that export's own `put` returned, taken from the event and checked to END in the derived key: neither is parsed, and building one here would mean guessing the bucket. The revision is not picked either, since a requeued job has more than one start: it counts the DISTINCT values and leaves NULL unless there is exactly one, because for a provenance field "cannot say" beats "the earlier one". Measured 2026-11-08: no export job records that field in any event, so all four recoveries get NULL and the operation reports it rather than writing a quiet one. Verification happens BEFORE the collapse, which the data forces: two jobs published the same digest for the same window under different `output_name` values, so the same bytes live under two keys, and collapsing first could keep the one whose object is gone. A count that matches is a coincidence waiting to happen; a set that matches is the population. Jobs publishing the same `(window, digest)` are one export run twice and collapse to the earliest; two publishing different ones are an ambiguity it refuses rather than resolves. A set with no candidate stays `legacy_undeclared` and is reported, which is the honest outcome for the one from the quick-annotation flow. A description somebody wrote is kept. `dry_run` writes nothing. Runs on `protea.jobs`.
-- `export_transitions_dataset` (`ExportTransitionsDatasetOperation`): writes the ADR-D56 transitions dataset's declared tables for named windows on a declared term axis. Separate from `export_research_dataset`, which keeps writing the D45 pair table: two datasets, and mixing them would make one `dataset` row unable to say which it is, so the names here carry a `transitions-` prefix. **Windows are named by id, never by a version pair**, because the pair stopped identifying a window (62 of 69 pairs carry more than one), and each is refused if it pivots on another axis than the one declared or was built by a revision other than the running one unless the payload names that revision; admitting them in bulk with a boolean would be the same as not checking. `cut` (the `ontology_snapshot_id` indexing `terms` and `term_ancestors`) and `cut_version` (the GOA release the laboratory's leakage rule compares against) are two fields and neither is derived from the other. It refuses a window whose `version_t_prime` is after `cut_version`, because its gain is in the model's future: the first run declared `cut_version` 211 over 64 windows and **15 of them ended after 211**, all inside the span the long validation window scores. Each window's truth, releases and horizon are read ONCE into a `WindowFacts`, since two loads are two chances for one bundle's tables to be about different reads, and a window missing a `source_version` or a publication date is refused rather than written as NULL (a NULL version does not fail the leakage rule, it skips it). Today it writes five of the ten: `window_terms_out`, the axis terms a window's rule leaves out; `residue_refs`, which names where each sequence's per-residue codes live (683,887 references over 5,344 shards, not one byte of code copied) and covers the WHOLE bank, a superset of any window selection, refusing a config with no rows because the rows are keyed by the VOCABULARY config and not by the model config it was derived from, which has zero; `terms` and `term_ancestors`, which are keyed by `cut` ALONE and so need no window list at all (43,558 live terms with their aspect and 595,072 closure pairs on the campaign axis, measured, against the 783,477 `Dag.closure`'s docstring quotes for a different release); and `queries`, whose population is `delta_accessions` imported from the target export rather than restated; both carry the full `evaluation_set_id` as the join key and the writer asserts the two tables agree about their window set before publishing either, because the loader joins them as text and a mismatch silently stops restricting the negatives. It **verifies the vectors before writing them: `terms` carries `vector_uri` and `vector_sha256` for the axis's text-embedding artefact, and the operation FETCHES those bytes and checks the digest before publishing either column, refusing when the artefact is absent, because a sha copied from a message is a locator for bytes nobody checked. It writes no `Dataset` row** while any of the ten tables is unwritten, since a half bundle with a sha is a bundle that lies. Asking for a table ADR-D56 declares but this revision does not write yet is refused rather than skipped. Runs on `protea.training`.
+- `export_transitions_dataset` (`ExportTransitionsDatasetOperation`): writes the ADR-D56 transitions dataset's declared tables for named windows on a declared term axis. Separate from `export_research_dataset`, which keeps writing the D45 pair table: two datasets, and mixing them would make one `dataset` row unable to say which it is, so the names here carry a `transitions-` prefix. **Windows are named by id, never by a version pair**, because the pair stopped identifying a window (62 of 69 pairs carry more than one), and each is refused if it pivots on another axis than the one declared or was built by a revision other than the running one unless the payload names that revision; admitting them in bulk with a boolean would be the same as not checking. `cut` (the `ontology_snapshot_id` indexing `terms` and `term_ancestors`) and `cut_version` (the GOA release the laboratory's leakage rule compares against) are two fields and neither is derived from the other. It refuses a window whose `version_t_prime` is after `cut_version`, because its gain is in the model's future: the first run declared `cut_version` 211 over 64 windows and **15 of them ended after 211**, all inside the span the long validation window scores. Each window's truth, releases and horizon are read ONCE into a `WindowFacts`, since two loads are two chances for one bundle's tables to be about different reads, and a window missing a `source_version` or a publication date is refused rather than written as NULL (a NULL version does not fail the leakage rule, it skips it). Today it writes six of the ten, including `gains` (the truth itself: which terms each query gained, with its stratum and the closure the gain is expressed in, 9.6M rows over the 65 admissible windows, sharing the query population as a guard rather than a filter so a gain for a protein `queries` excludes is refused rather than dropped); `window_terms_out`, the axis terms a window's rule leaves out; `residue_refs`, which names where each sequence's per-residue codes live (683,887 references over 5,344 shards, not one byte of code copied) and covers the WHOLE bank, a superset of any window selection, refusing a config with no rows because the rows are keyed by the VOCABULARY config and not by the model config it was derived from, which has zero; `terms` and `term_ancestors`, which are keyed by `cut` ALONE and so need no window list at all (43,558 live terms with their aspect and 595,072 closure pairs on the campaign axis, measured, against the 783,477 `Dag.closure`'s docstring quotes for a different release); and `queries`, whose population is `delta_accessions` imported from the target export rather than restated; both carry the full `evaluation_set_id` as the join key and the writer asserts the two tables agree about their window set before publishing either, because the loader joins them as text and a mismatch silently stops restricting the negatives. It **verifies the vectors before writing them: `terms` carries `vector_uri` and `vector_sha256` for the axis's text-embedding artefact, and the operation FETCHES those bytes and checks the digest before publishing either column, refusing when the artefact is absent, because a sha copied from a message is a locator for bytes nobody checked. It writes no `Dataset` row** while any of the ten tables is unwritten, since a half bundle with a sha is a bundle that lies. Asking for a table ADR-D56 declares but this revision does not write yet is refused rather than skipped. Runs on `protea.training`.
 
 These three operations form the **post-reranker functional-enrichment stage, the last component to be developed before uploading results to LAFA**. All three are fully wired and registered in the `OperationRegistry`; the remaining prerequisite is an InterProScan binary install on the host.
 

docs/source/adr/D56-transitions-dataset-is-tables-keyed-by-what-they-depend-on.rst

@@ -330,6 +330,40 @@ constraint this section states is only the bound: for a bundle that must not
 leak into the long validation window, ``cut_version`` has to be at or before
 the release that window opens on.
 
+``gains`` shares the query population, and that is a guard
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Noted 2026-11-08, when the table was written. ``queries`` excludes a protein
+with no stored sequence, because the exported FASTA excludes it and the
+training rows have to be about the scored population. So a gain for a protein
+outside that set would be a gain for a query the loader does not know, and the
+two tables would disagree about who the queries are **without either being
+wrong on its own**.
+
+The writer therefore computes the query population once and refuses a gain
+outside it, rather than dropping one. Dropping would make ``gains`` quietly
+smaller than the truth it copies, and a row count is the only thing a reader
+could compare it against. Measured over the 65 admissible windows: **zero**
+delta proteins lack a sequence, so the guard changes nothing today and says so
+when that stops being true. The same shape covers a gained term the axis cannot
+name: the loader groups by aspect, so such a row is one it cannot place.
+
+**The closure is derived from the truth profile in one place.** ``projected``
+means the gain is expressed on the pivot and ``as_closed`` on each end's own
+graph, and the same window under two closures is two different truths. A
+profile with no declared closure is refused rather than guessed, and a test
+asserts that every value of ``TRUTH_PROFILES`` has one, so adding a profile
+forces someone to say which graph it closes in at the moment it is declared
+rather than at run time.
+
+**The shape, measured before the code was written.** Over those 65 windows
+``gains`` is **9.596.563 rows**, the largest window 691.553 and the mean
+147.639. Six short columns at that count is the one place in this dataset where
+building a tuple per row would hurt, so the columns are six parallel lists
+appended to per window, and the order is stable by construction rather than by
+a final sort: windows in order, strata in their declared order, queries sorted,
+terms sorted.
+
 The window is the join key, and it is the full id
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -372,6 +406,21 @@ describes and nothing would see it. Measured over the 131 evaluation sets in
 the database, two values are enough and a third is not needed: **122 are
 ``native`` and 9 are ``pivot``**.
 
+**AND THE WRITER MADE EXACTLY THAT MISTAKE, twice, on 2026-11-08.** The first
+version of ``gains`` mapped ``projected`` to ``pivot`` and ``as_closed`` to
+``native``, in a constant named ``_CLOSURE_OF_PROFILE``, with a test that
+walked the values of ``TRUTH_PROFILES`` and therefore could not see it. The
+laboratory had made the same mapping a day earlier and withdrawn it. Both times
+the paragraph below was already in this record, written by an earlier PR of
+this same project: the warning was not missing, it was not read.
+
+So the derivation now compares the two natives against the pivot and against
+each end's corpus ontology, the way the window inventory already does, a third
+combination is refused rather than named, and the test walks combinations of
+NATIVES. The case that settles it is in the database: the twins of
+``160 -> 177`` are both ``projected`` and differ only here, so under the
+mapping the (B) one would have been written as ``pivot``.
+
 **It is NOT ``truth_profile``.** That field says how the truth is derived,
 ``projected`` or ``as_closed``; ``closure`` says which graphs the two ends
 close in. They vary independently, and the probe that settles the open decision