You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Raised in review of #706 (vahid-ahmadi): the UK input-mass parity gate's shared relative_tolerance is minted honestly as the worst surviving measured drift, but that edge is currently set by a column whose deviation is already explained — so the fence grants every other column a much larger unexplained-regression budget than the compared population actually needs.
The measurement (full surface, recorded in experiments/630-uk-gate-adjudication-receipts.md)
128 columns compared against the efrs-post-calibration reference at floor 0.0. The compared population sits at median 0.24, p75 0.42, p90 0.63, p95 1.15 |relative drift|, while the committed tolerance is 4.5218. Only five columns exceed 1.5. So the gate today is a gross mass-loss/explosion fence, not a regression detector: a 2× mass move in a mid-band column passes silently.
The decay above the bulk is smooth, with no natural cut point — ranks 2–8: is_enhanced_disabled_for_benefits 4.5218, sda_reported 3.0587, jsa_income_reported 2.1690, lump_sum_income 1.5237, education_grants 1.1557, bsp_reported 1.1509, non_residential_property_value 0.9026.
The instrument already exists there — per-reference reviewed exclusions with receipts, expiry, and stale-fail — and re-minting the tolerance is a parameter edit under the same reviewed mechanism. What is missing is evidence per column, and the register deliberately refuses unevidenced reasons (reason, approved_by, adjudication, approved_on, expires_on).
Today exactly one of the top drifts carries a documented explanation: #703's head-to-head receipt verified the disability categories and flags agree row-for-row at 100.0000% with the incumbent's own create_frs at the pinned revision, attributing the residual delta against the frozen reference to the artifact's vintage (rev 655dd07e predates the disability-logic fixes) plus the SPI-synthetic composition (~37% of persons), with re-measurement scheduled at matching composition at E10. Excluding only that column re-mints at 3.0587 — a 1.48× gain that does not change the gate's practical sensitivity.
sda_reported and jsa_income_reported are plausibly the same compositional story (FRS-reported legacy-benefit columns under a shifted benefit population), but carry no verification of their own.
The work
For each of the top ~7 drifts, establish whether the deviation traces to the known comparability causes (reference vintage; SPI-synthetic composition) or is a genuine candidate-side deviation — the Port the FRS derived layer as declarative source stages #703 head-to-head is the methodological template.
Receipt the explained ones into input_mass_reviewed_exclusions.json under efrs-post-calibration, expiring at the E10 recomposition so the register is forced back open exactly when the composition story changes.
Re-mint relative_tolerance at the worst unexplained survivor, with the receipt naming the surviving edge column.
Consider whether an explained-drift entry should be a distinct register class from a suppression entry — an exclusion removes all regression detection for that column, which is a real cost the current single-class register does not express.
Note the asymmetry in (4): every column moved into the register trades "no detection at all on that column" for "better detection on the remaining 127". That tradeoff, not just the tolerance number, is what the triage should decide.
An alternative worth weighing at the same time: per-column tolerance bands (most precise, ~128 parameters, high re-mint churn) versus the shared tolerance plus register (coarse, cheap, what exists today).
Refs #706 (review thread and the receipts doc), #703 (the receipt precedent), #609/#610 (the measurement doctrine), #665 (E10 recomposition).
Raised in review of #706 (vahid-ahmadi): the UK input-mass parity gate's shared
relative_toleranceis minted honestly as the worst surviving measured drift, but that edge is currently set by a column whose deviation is already explained — so the fence grants every other column a much larger unexplained-regression budget than the compared population actually needs.The measurement (full surface, recorded in
experiments/630-uk-gate-adjudication-receipts.md)128 columns compared against the
efrs-post-calibrationreference at floor 0.0. The compared population sits at median 0.24, p75 0.42, p90 0.63, p95 1.15 |relative drift|, while the committed tolerance is 4.5218. Only five columns exceed 1.5. So the gate today is a gross mass-loss/explosion fence, not a regression detector: a 2× mass move in a mid-band column passes silently.The decay above the bulk is smooth, with no natural cut point — ranks 2–8:
is_enhanced_disabled_for_benefits4.5218,sda_reported3.0587,jsa_income_reported2.1690,lump_sum_income1.5237,education_grants1.1557,bsp_reported1.1509,non_residential_property_value0.9026.charitable_investment_giftsonly)Why this was deferred out of #706
The instrument already exists there — per-reference reviewed exclusions with receipts, expiry, and stale-fail — and re-minting the tolerance is a parameter edit under the same reviewed mechanism. What is missing is evidence per column, and the register deliberately refuses unevidenced reasons (
reason,approved_by,adjudication,approved_on,expires_on).Today exactly one of the top drifts carries a documented explanation: #703's head-to-head receipt verified the disability categories and flags agree row-for-row at 100.0000% with the incumbent's own
create_frsat the pinned revision, attributing the residual delta against the frozen reference to the artifact's vintage (rev655dd07epredates the disability-logic fixes) plus the SPI-synthetic composition (~37% of persons), with re-measurement scheduled at matching composition at E10. Excluding only that column re-mints at 3.0587 — a 1.48× gain that does not change the gate's practical sensitivity.sda_reportedandjsa_income_reportedare plausibly the same compositional story (FRS-reported legacy-benefit columns under a shifted benefit population), but carry no verification of their own.The work
input_mass_reviewed_exclusions.jsonunderefrs-post-calibration, expiring at the E10 recomposition so the register is forced back open exactly when the composition story changes.relative_toleranceat the worst unexplained survivor, with the receipt naming the surviving edge column.Note the asymmetry in (4): every column moved into the register trades "no detection at all on that column" for "better detection on the remaining 127". That tradeoff, not just the tolerance number, is what the triage should decide.
An alternative worth weighing at the same time: per-column tolerance bands (most precise, ~128 parameters, high re-mint churn) versus the shared tolerance plus register (coarse, cheap, what exists today).
Refs #706 (review thread and the receipts doc), #703 (the receipt precedent), #609/#610 (the measurement doctrine), #665 (E10 recomposition).
🤖 Generated with Claude Code