Skip to content

Retarget the UK build to FRS 2024-25: re-pin raw vintages, regenerate parity instruments, re-measure gate baselines #723

Description

@juaristi22

Workstream E of the UK-migration master epic #665, child of #145. Starts as soon as #681 (E5) and #682 (E6) land; prerequisite for verifying #684 (E8) at the right vintage and for #686 (E10) parity.

Decision recorded (2026-08-19)

The migration targets FRS 2024-25 directly (survey/base year 2024, calibration year 2025) — the incumbent's live production line (enhanced_frs_2024_25, v1.56.x). The 2023 vintage build is deferred (#687 rescoped): parity at a vintage consumers have already moved past would buy a second parity pass later for no independent value. Donor-survey vintages are unchanged — the incumbent's 2024-25 build uses the same SPI 2022-23, WAS round 8, LCFS 2023-24, and ETB files already staged locally; only the FRS spine and the target year move.

This child is deliberately mechanical — the tooling for every step already exists; the work is re-pointing and re-running, plus adjudicating what the re-runs surface.

Scope

  1. Raw-vintage audit at 2024-25 (the E2/U7 pattern): sha+size-pin the FRS 2024-25 tab set (25 tables, staged locally), and confirm the stage declarations accommodate the known 2024-25 schema quirks the incumbent handles explicitly — the household table is not sernum-ordered (the weight-scrambling sort bug), the salary-sacrifice asked-state fields, and any EDUCQUAL/derived-field deltas surfaced by the audit.
  2. Re-pin stage manifests + adopt the release-object discipline (moved forward from the old E11): one frozen declarative release object naming survey/base/calibration years, UKDS study number, DOI, zip sha; tape years labeled by tax year (2024_25) per the SPI-stage convention.
  3. Regenerate the parity instruments from enhanced_frs_2024_25 (pull + pin revision and sha; requires HUGGING_FACE_TOKEN): efrs_parity_reference, the known-gaps register (re-adjudicated — 2023 gaps may close, new ones may appear and must be signed, not inherited), and the record-count identity re-derived at the new vintage.
  4. Re-verify the landed stages (E1–E7) at the new vintage: re-run each per-stage verification harness against the re-frozen bar; land fixes for any vintage-sensitive declaration.
  5. Re-measure the weighted-integrity baselines at the new vintage (tools/measure_uk_weighted_integrity_baselines.py): weight-ratio reviewed maximum, ESS fraction, zero-weight strata, and the input-mass reference. These become the candidate thresholds Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686 arms.

Out of scope

Acceptance

  • The full ingest StagePlan runs at smoke rung on the 2024-25 tabs.
  • Regenerated parity instruments committed, pinned to the incumbent revision/sha they were extracted from.
  • Per-stage verification green for E1–E7 at the new vintage; every divergence either fixed or signed in the known-gaps register.
  • New-vintage integrity baselines recorded and referenced from the Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686 scope.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions