Add DWP Universal Credit caseload facts

[MaxGhenis]

Summary

Adds the facts-only DWP source package for Universal Credit deductions statistics March 2025 to February 2026 as the UC slice of microcosm#701, following the chronicle#166 boundary ruling and the claim on microcosm#622.

The package records:

• all 12 published monthly counts of UC units with a deduction;
• all 12 published monthly shares;
• 11 monthly total UC benefit-unit counts from April 2025 to February 2026, derived within each publisher row as count divided by share and rounded to 10,000, with both source cells retained in lineage.

DWP's “household” is the UC unit of assessment, represented here as benefit_unit, not an ONS household. This follows the entity ruling documented in policyengine-uk-data#457.

The source-package schema has no separate period-aggregate fact surface, so the approximate calendar-2025 average and the December-to-February plateau remain consumer aggregations. packages/obr/efo_expenditure_march_2026/source_package.yaml already carries OBR Universal Credit expenditure facts, including FY2025–26 in-cap and outside-cap lines; this PR does not duplicate them. db/etl_obr.py is a separate legacy ETL and does not carry those rows.

Verification

Every Table 1 input was checked against /Users/maxghenis/PolicyEngine/_drafts/wpi-followup-2026-08-05/data/table_1.csv before authoring.

Formatter and scoped tests:

All checks passed!
........                                                                 [100%]
8 passed, 12 warnings in 1.17s

The warnings are existing third-party PyIceberg/Pydantic deprecations.

Deviations

• No calendar-year or plateau aggregate fact was added because the source-package schema has no governed home for it.
• No OBR fact was added because the existing OBR source package already contains the relevant UC expenditure series.
• Chronicle has no changelog-fragment convention or changelog.d directory, so no fragment is introduced.

[juaristi22]

All findings from the adversarial review have been addressed and pushed in 25ca569 after rebasing the PR branch onto current main (9553be0).

Fixed:

• Preserved legacy record_set_spec_hash values by omitting unset divisor_column and round_to fields from the hash payload.
• Included divisor_column cells in source_regions_from_record_set_spec, so derived total_units source regions now cover the full lineage range.
• Added DWP storage.r2 manifest provenance so the new UC deductions facts satisfy the consumer contract in package and merged bundle exports.
• Updated bundle goldens for the newly included DWP package and its 35 benefit_unit facts.

Verification:

• uv run pytest -q tests/test_chronicle_bundle.py::test_build_bundle_writes_merged_consumer_contract
• uv run pytest -q tests/test_chronicle_source_package.py::test_dwp_uc_deductions_package_preserves_rows_and_derives_uc_units tests/test_chronicle_source_package.py::test_source_package_alias_compiles_soi_table_1_1_specs tests/test_chronicle_source_package.py::test_empty_guard_cells_do_not_change_legacy_single_cell_hash
• uv run ruff check chronicle policyengine_chronicle db scripts tests

[vahid-ahmadi]

Reviewed as the facts half of the UC chain (with microcosm#727), since that's the pair that closes the UC row on PolicyEngine/microcosm#731.

The derivation is well-declared. Deriving total UC benefit units as deductions-count ÷ published-share out of Table 1 is a real inference rather than a published figure, and the package says so where it counts: concept_relation: source_label, the evidence URL pinned to the specific DWP release, and concept_evidence_notes carrying both the arithmetic and the DWP-household-means-UC-benefit-unit disambiguation with the uk-data#457 pointer. round_to: 10000 is the right honesty about a ratio-derived count — at ~6.7m units that's 0.15%, well inside the share's own precision, and it stops the derived value from implying digits the source doesn't support. Emitting the deduction count, the share, and the derived total as three separate concepts rather than only the answer keeps the derivation auditable from the bundle alone.

@juaristi22's hash-payload fix is the load-bearing one for the repo rather than for this package: omitting unset divisor_column/round_to from record_set_spec_hash preserves every legacy spec hash, so a schema extension doesn't silently re-cut identities across the existing corpus. Extending source_regions_from_record_set_spec to include divisor cells is the matching correctness half — a derived value's lineage has to cover the cells it actually divides by.

One question that belongs on this side of the seam: is a point-in-time monthly stock the basis calibration wants?

The package emits twelve monthly facts (2025-03 … 2026-02). On the consumer side those resolve by "latest at or before the target period," so a national UC caseload target binds a single month — December 2025 at the #723 retarget's calibration year (details on microcosm#727). UC caseload trends and carries seasonality, so a single month is a different quantity from the year's level, and nothing in either artifact currently says which one is intended.

If the intended basis is annual, this package is the natural place to fix it: an explicit annual concept (mean of the twelve months, or the fiscal-year average) derived and declared here would give the consumer something unambiguous to pin, and would carry its own evidence note the way total_units already does. If a point-in-time stock is genuinely what's wanted, then the consumer should pin the month explicitly — either way the choice should be written down rather than emerging from a tie-break rule. Not blocking this PR: the monthly facts are correct and useful regardless of which basis the calibration ends up binding.

[juaristi22]

Thanks, Vahid. I agree the basis choice needs to be explicit, but I think that operation should live in Microcosm rather than Chronicle.

Chronicle needs the facts to stay as published, with provenance preserved all the way back to the source release. Turning the monthly published stock series into a calibration-specific annual or pinned-period target would make this package carry consumer semantics instead of just the auditable fact lineage.

That choice is being adjudicated on the Microcosm side here: PolicyEngine/microcosm#701 (comment). Now that this Chronicle PR is merged, Microcosm #727 can consume the published facts and encode the calibration basis there once CI is green.