Consume Ledger UK facts: uk_target_references, mapping module, and a compile-parity gate

[juaristi22]

Depends on: PolicyEngine/chronicle#132 (parity surface) for the gate fixture; PolicyEngine/chronicle#133 + PolicyEngine/chronicle#134 and the uk_national profile for full coverage. Consumption code can land behind fixtures before all waves finish.

Context

Populace's Ledger consumption layer is built and proven on the US side:

• populace/build/ledger_artifact.py — hash-verified consumer-artifact loading (duck-typed, stdlib-only)
• populace/build/ledger_targets.py — LedgerTargetMapping / LedgerTargetReference / compile_ledger_target_references (identity-only value resolution; uprating/aging as populace-side declarations)
• populace/build/us/fiscal_target_references.json — 11 references, including counterfactual targets via metadata.kind: neutralize_variable
• tools/build_us_fiscal_refresh_release.py — --ledger-facts + --ledger-facts-sha256 / --ledger-manifest-sha256 pinning into the release manifest

The UK has exactly one Ledger consumer today (firm generation, uk_firm_source_data_from_ledger_facts) plus profile-aware local metrics (populace.build.uk.local_targets, #158). There is no UK household-sector target compilation from Ledger. This issue adds it, with a parity gate proving the Ledger-backed surface reproduces what production calibrated against.

Doctrine boundaries this issue must respect: Ledger owns facts; populace owns the active subset (porting a vintage never activates it), model-variable mapping, period alignment/uprating declarations, and all reconciliation/aging. value_operation stays identity.

Work items

1. populace/build/uk/uk_target_references.json + loader — mirror of the US shape: one LedgerTargetReference per active UK target (entity, measure/filter mapping, family, period, selector by record_set_id/measure). Counterfactual and cross-tab targets (two-child limit, UC family-type splits, benefit cap) use metadata.kind + a UK metric-provider registry, following the JCT neutralize_variable pattern — the reference declares, the provider computes model-side.

2. UK mapping module (populace/build/uk_runtime/ledger_targets.py or equivalent) — LedgerTargetMapping for UK concepts: measure_by_concept / measure_by_source_record_id → policyengine-uk variables, filter_by_domain, entity_by_ledger_entity (person/benunit/household), family_by_source_name (obr/dwp/hmrc/ons/slc/voa/…). Unmapped facts are reported as unsupported, never silently dropped.

3. Profile consumption — load uk_national (and keep uk_local_geography consumption aligned) through the existing profile loader; compile to TargetSpec rows via the mapping.

4. Compile-parity gate (the acceptance instrument):

   • Fixture A (production surface): the 149 rows in calibration_diagnostics.json of release populace-uk-2023-dd68c73-4aa4b14-20260619T023711Z (policyengine/populace-uk-private) — every (name, period, value) must resolve from the pinned Ledger artifact.
   • Fixture B (current registry): policyengine-uk-data's registry compile at pinned ref ebf733c for the current calibration year — Ledger-backed compile must match, with every intentional difference (vintage moved, target dropped/renamed) enumerated in a signed exclusion list. No silent gaps in either direction.
   • Gate output follows the existing GateResult conventions; wire into the UK release preflight beside the input-coverage gates.

5. Release-manifest provenance — UK build/release manifests record ledger_facts_sha256 (+ manifest sha), same as the US fiscal refresh, so a release pins exactly which Ledger data resolved its targets.

6. CGT single-homing — retire the hand-maintained UK_CGT_TARGET_SPECS in populace/build/uk_runtime/fiscal_targets.py onto references against the UK source packages, wave 2: national fiscal/benefit families + uk_national target profile chronicle#133 hmrc/cgt_statistics package (keep the coverage-requirement machinery; only the values' home changes).

Explicitly out of scope

• Running a calibration. The UK national build performs no calibration today by design (tools/build_uk_national_dataset.py: the staging path "performs no calibration and therefore has no real target-surface or target-fit evidence"). Wiring the compiled registry into a calibration seam is Calibrate the UK national build from Ledger-backed targets #623.
• Any Ledger-side authoring.

Acceptance

• uk_target_references.json compiles through compile_ledger_target_references against a hash-pinned UK consumer artifact with zero unsupported-fact surprises (unsupported list is empty or explicitly signed).
• Parity gate green on Fixture A; Fixture B differences fully enumerated.
• A UK release manifest carries the Ledger pin fields.
• CGT specs resolve from Ledger with unchanged downstream coverage requirements.

---

Status update (2026-08-18, from #707)

Item 1 is done by #707 (per the scope adjudication): the re-homed contract uk/uk_national_targets.json (186 declarations, exact 651-row parity closure: 609 mapped + 42 excluded + 3 signed unmapped declarations) plus the typed uk/target_references.json first active subset — 13 references at period 2023 — and the ledger_targets.py selector extension (dimension_values strict-typed pins, list-form dimensions, list membership, record_set_id/groupby_dimension aliases). One resolver now serves the US, Belgian, and UK surfaces (#708).

Findings that reshape the remaining items:

• Fixture-B year decision needed: the incumbent's effective surface at its current calibration year 2025 is 636 rows, not 651 — the 15 dwp/uc/two_child_limit/* rows carry only {2026} values and _resolve_value refuses forward extrapolation. Recommend Fixture B at 2026 → 651 (the full surface; avoids codifying the degraded year) with an explicit 636@2025 caveat. The contract's registry_parity.accounting_notes records this.
• Never trust a live incumbent run as a parity instrument: create_target_matrix swallows per-target failures with a bare except + warning (no column-count assertion anywhere), and its surface is already degraded at the pin — the salary-sacrifice IT-relief source CSV is 410-Gone, silently suppressing rows whose dispatch branches remain as dead code. Fixture B must compile from the registry definitions, not from a network-dependent run.
• Multi-fact sum resolution is the largest resolver gap (23 deferred targets): band/groupby targets are sums over matched fact sets (resolution_defaults.operation: "sum"); the shared compile path resolves exactly one fact per reference. The CGT totals also expose that _selector_period_invariant_key does not strip UK record-set year spellings (37 same-series year-rows fail to collapse).
• Country-level geography ambiguity (49 deferred targets): UK-wide (K02000001) and the four constituent countries share level country; per-target geography_id pinning is the adjudication.
• No fact ≤ 2023 (101 deferred targets): all OBR EFO lines and the 2025/2026-vintage DWP/VOA/Scotgov/SLC families. Activation grows via this issue's uprating/period declarations (uprating_index fields exist on LedgerTargetReference) or with the WS-E/F year move.
• The per-target classification lives in Adopt the UK national calibration contract (re-homed from chronicle#164 per the #166 ruling) #707's authoring-run membership report (attached to the PR discussion); the deferred classes above partition all 173 non-active targets.

[juaristi22]

Shape adjustment from the chronicle#166 ruling (selection contracts live in the consumer; Chronicle is facts-only):

• Work item 3 ("load uk_national … through the existing profile loader") is superseded: there is no Chronicle-side profile. The contract now lives in this repo as uk/uk_national_targets.json (Adopt the UK national calibration contract (re-homed from chronicle#164 per the #166 ruling) #707) — 186 value-free target declarations with ledger_selector (including dimension_values pins the reference resolver must honor), measurement contracts, policyengine bindings with the declarative counterfactual kinds, and per-target assertion policies.
• Work item 1's shape is unchanged in spirit: uk_target_references.json (or compiling directly from Adopt the UK national calibration contract (re-homed from chronicle#164 per the #166 ruling) #707's resource) declares the active subset; the contract resource carries the full portable surface plus the 651-row registry-parity accounting (609 mapped + 42 signed exclusions at ebf733c) — Fixture B for work item 4's compile-parity gate is embedded in the resource.
• uk_local_geography consumption should also stop duck-typing a Ledger export once Migrate the UK local-geography and firms contracts out of Chronicle (chronicle#166 ruling) #708 migrates that contract in-repo.

Fact-side note: the MYE sex-split record sets backing the 36 sex×age targets are landing as a facts-only chronicle PR.