Ingest the full raw FRS table set and assemble the UK spine as a Frame

[juaristi22]

Workstream E2 of the UK migration (#145, master epic #665). Closes #678. Stacked on #690 (E1) — review this PR's own diff as the last commit; the base is the E1 branch.

What

Microcosm now assembles the UK spine Frame itself from the 14 raw FRS 2023-24 tables — the future replacement for loading the certified incumbent H5 (--input-h5). Scope is deliberately fenced to direct raw→column mappings: no derived layer (E3), no stochastic layer (E4), no imputations (E5+), and neither incumbent mid-build Microsimulation call site.

• uk/source_stages.json — new frs_spine stage, first in the manifest: 14 sha256+size-pinned licensed tab artifacts (tax-year vintage label 2023_24), six operations reusing generic kinds plus three new generic ones (assemble_group_entities, map_coded_amounts, annualize_periodic_amounts), and an 82-column load-bearing output surface (outputs compile into StagePlan.produces — the E1 licensed-data lesson applied from the start). The two E1 stage declarations are untouched (lockstep test still pins them).
• country_stage_plan(..., stage_names=...) — country-agnostic subsetting so two pipelines (spine build; national staging) can share one manifest: default keeps all-stage behavior, explicit tuples are validated (non-empty, no duplicates, unknown names refused) and execute in manifest order. The national driver now selects its two stages explicitly.
• uk_runtime/frs_spine.py — pinned verify-then-open reads, the incumbent's entity/ID scheme (household-ID sort invariant ported as regression armor), gross4 → typed design weights, and the raw mappings: demographics, incomes (pension/accounts/oddjob tables), the full benefit-code table including the JSA/ESA contribution/income splits, expenses, and the household housing surface. WEEKS_IN_YEAR applied uniformly — a signed difference at the three incumbent *52 call sites.
• tools/build_uk_frs_spine.py — fail-closed driver: atomic write through the shared writer, .build.json evidence sidecar (aggregates only), a Logbook row per attempt (pipeline uk-frs-spine), --emit-nonzero-shares parity mode, and no seed (the build is structurally deterministic; documented in --help).

UK-specificity note

UK-specific: the stage declaration (FRS/UKDS identities), frs_spine.py (FRS coded domains, benefit codes, UK entity structure), the driver's pipeline identity, and the domain maps (region/tenure/accommodation/band). General: the subsetting extension, the three new operation kinds (any country can use them), and everything routed through the shared manifest/StagePlan/writer machinery.

Licensed-data acceptance (all data stayed local; disclosure-safe evidence only)

• Spine built from the real tabs: 36,248 persons / 19,374 benunits / 16,754 households; gross4 weight total 28,782,960 — right magnitude against the certified spine's mass log.
• Determinism on real data: two independent builds, compare_uk_h5_payload.py verdict payload_identical: true (exit 0).
• Nonzero shares vs the committed parity reference: 63 of the spine's 82 columns overlap the reference's 145; only 8 columns differ by more than 0.02, and every one has a known mechanism — the SPI stack and dividend re-imputation (E7), the salary-sacrifice conversion (E8), and stack/clone composition dilution. No degenerate or missing raw mapping.
• Both build attempts landed hash-chained Logbook rows.
• Open adjudication flagged: the raw read yields 16,754 households vs the incumbent base's 16,364 at the same vintage — a 390-household composition rule to adjudicate at the assembly seam (E7/E10), not an E2 blocker (E2's contract is raw fidelity).

Implementation notes

Codex implemented under /codex-implement-plan; deviations recorded in review: byte-identical output shas are unattainable through the shared H5 writer (HDF5 metadata timestamps — the known reason compare_uk_h5_payload.py exists), so determinism is asserted at payload level; the vintage-conditional incumbent branches are resolved at 2023-24 and enumerated in the report (post-2021 domestic-rates branch; corrected contiguous 12-region map). One integration fix landed in review: the E1 shim-equivalence test selected manifest stages positionally and broke when frs_spine became the manifest root — it now selects by name and passes the explicit two-stage subset.

🤖 Generated with Claude Code