Skip to content

US county administrative calibration packages - #185

Merged
MaxGhenis merged 1 commit into
mainfrom
county-admin-packages
Aug 20, 2026
Merged

US county administrative calibration packages#185
MaxGhenis merged 1 commit into
mainfrom
county-admin-packages

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

First sub-state calibration facts through the provenance-classed pipeline: IRS SOI county returns and AGI (administrative, 3,143 counties, values cross-checked against the raw publisher CSV), Census PEP Vintage 2024 county population (census, 3,144 county equivalents), and USDA SNAP FY2025 monthly state caseloads (administrative, 636 publisher-backed facts). Real publisher bytes fetched per the offline fetch manifest and hash-pinned; packages deterministically regenerated by chronicle.sources.admin_packages (fixture markers removed; the 3,000-4,000 county inventory guard enforced); consumer contract valid with content-addressed R2 provenance declared. The fixture masquerade guards now build their fixture inputs in-test, so they hold regardless of the repo db/data state.

Rebased over the wave-3 merges: kept main's header_row preamble machinery in sources/rows.py and swapped in the cp1252-tolerant decode_delimited_text (Census county files are ANSI); merged-bundle pins re-measured from a fresh build_bundle run over the union (155,288 facts).

Note: the live FNS SNAP publisher URL now serves different bytes than the archived vintage - recorded as note_new_vintage_sha256 in FETCH-MANIFEST.json for a future vintage package; archived bytes retained. Sibling of the Eurostat pilot PR #168.

🤖 Generated with Claude Code

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Adversarial review (gpt-5.6-sol, ultra) blocked the first push with three findings, all confirmed against the code and now fixed:

  1. Connecticut boundary identity (high). The IRS county file reports Connecticut's nine planning-region county equivalents (09110-09190), which only exist from the 2022 boundary vintage - but the generator stamped every row 2020_census, giving 18 facts a nonexistent boundary identity inside their stable keys. Fixed: the IRS county record set and rows now carry geography_vintage: 2022 (matching the PEP county convention of vintage-year labeling), and the production test pins the Capitol Planning Region facts against raw publisher bytes (492,650 returns; $47,178,097,000 AGI) plus asserts the CT geoid set is exactly the nine planning regions.

  2. Permissive _county_fips (medium/high). int()+zfill accepted signed and overwide inputs (("100","001") -> "100001", ("-1","001") -> "-1001"). Now structurally validated: 1-2 digit state, 1-5 digit county, canonical five-digit result; eight malformed shapes covered by parametrized rejection tests. Also added the marker-removal masquerade test: stripping fixture: true from a small synthetic source trips the 3,000-4,000 inventory guard.

  3. Broken/integrity-hostile reproduction metadata (medium). generated_by and the fetch workflow referenced the removed ledger.sources.admin_packages module - fixed to chronicle.sources.admin_packages throughout, packages regenerated (zero drift). The county post-download steps no longer instruct replacing established hash pins from freshly downloaded mutable bytes; they now verify against the pin and STOP on mismatch (new bytes = new reviewed artifact). Stale SOL-D2A scaffolding notes removed from the tree.

Full suite green locally after the fixes.

@MaxGhenis
MaxGhenis force-pushed the county-admin-packages branch from 44ff267 to 20a9d12 Compare August 20, 2026 03:56
@MaxGhenis
MaxGhenis merged commit b014c38 into main Aug 20, 2026
2 checks passed
@MaxGhenis
MaxGhenis deleted the county-admin-packages branch August 20, 2026 04:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant