Skip to content

Tighten the UK input-mass tolerance by receipting the explained top drifts #713

Description

@juaristi22

Raised in review of #706 (vahid-ahmadi): the UK input-mass parity gate's shared relative_tolerance is minted honestly as the worst surviving measured drift, but that edge is currently set by a column whose deviation is already explained — so the fence grants every other column a much larger unexplained-regression budget than the compared population actually needs.

The measurement (full surface, recorded in experiments/630-uk-gate-adjudication-receipts.md)

128 columns compared against the efrs-post-calibration reference at floor 0.0. The compared population sits at median 0.24, p75 0.42, p90 0.63, p95 1.15 |relative drift|, while the committed tolerance is 4.5218. Only five columns exceed 1.5. So the gate today is a gross mass-loss/explosion fence, not a regression detector: a 2× mass move in a mid-band column passes silently.

The decay above the bulk is smooth, with no natural cut point — ranks 2–8: is_enhanced_disabled_for_benefits 4.5218, sda_reported 3.0587, jsa_income_reported 2.1690, lump_sum_income 1.5237, education_grants 1.1557, bsp_reported 1.1509, non_residential_property_value 0.9026.

columns receipted into the register re-minted tolerance tightening
1 (today: charitable_investment_gifts only) 4.5218
2 3.0587 1.48×
4 1.5237 2.97×
7 0.9026 5.01×
20 0.5167 8.75×
30 0.4401 10.27×

Why this was deferred out of #706

The instrument already exists there — per-reference reviewed exclusions with receipts, expiry, and stale-fail — and re-minting the tolerance is a parameter edit under the same reviewed mechanism. What is missing is evidence per column, and the register deliberately refuses unevidenced reasons (reason, approved_by, adjudication, approved_on, expires_on).

Today exactly one of the top drifts carries a documented explanation: #703's head-to-head receipt verified the disability categories and flags agree row-for-row at 100.0000% with the incumbent's own create_frs at the pinned revision, attributing the residual delta against the frozen reference to the artifact's vintage (rev 655dd07e predates the disability-logic fixes) plus the SPI-synthetic composition (~37% of persons), with re-measurement scheduled at matching composition at E10. Excluding only that column re-mints at 3.0587 — a 1.48× gain that does not change the gate's practical sensitivity.

sda_reported and jsa_income_reported are plausibly the same compositional story (FRS-reported legacy-benefit columns under a shifted benefit population), but carry no verification of their own.

The work

  1. For each of the top ~7 drifts, establish whether the deviation traces to the known comparability causes (reference vintage; SPI-synthetic composition) or is a genuine candidate-side deviation — the Port the FRS derived layer as declarative source stages #703 head-to-head is the methodological template.
  2. Receipt the explained ones into input_mass_reviewed_exclusions.json under efrs-post-calibration, expiring at the E10 recomposition so the register is forced back open exactly when the composition story changes.
  3. Re-mint relative_tolerance at the worst unexplained survivor, with the receipt naming the surviving edge column.
  4. Consider whether an explained-drift entry should be a distinct register class from a suppression entry — an exclusion removes all regression detection for that column, which is a real cost the current single-class register does not express.

Note the asymmetry in (4): every column moved into the register trades "no detection at all on that column" for "better detection on the remaining 127". That tradeoff, not just the tolerance number, is what the triage should decide.

An alternative worth weighing at the same time: per-column tolerance bands (most precise, ~128 parameters, high re-mint churn) versus the shared tolerance plus register (coarse, cheap, what exists today).

Refs #706 (review thread and the receipts doc), #703 (the receipt precedent), #609/#610 (the measurement doctrine), #665 (E10 recomposition).

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions