Repository navigation
Keep paired confidence statistics JSON serializable - #312
Open
kocean9-freedom wants to merge 1 commit into
Open
kocean9-freedom wants to merge 1 commit into
kocean9-freedom wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Paired bootstrap results for chrF and TER cannot be printed with
--format json:print_results_tableraisesTypeError: Object of type float32 is not JSON serializable. The same failure occurs throughPairedTest(..., test_type='ar', n_ar_confidence=...)followed by the shared JSON formatter.The statistics are accumulated as float32 arrays, and
estimate_cireturns NumPy scalars. These are stored unchanged inResult.meanandResult.ci, then passed tojson.dumpsthroughResult.__dict__. Return Python floats fromestimate_ci, as its annotation already specifies. The conversion happens after the existing calculation and preserves its numeric values.The regression runs the real paired tests and shared output formatter for BLEU, chrF and TER, under both AR with confidence intervals and BS. Before the fix, the four chrF/TER cases raise the serialization error and both BLEU controls pass. Afterward all six pass. This is independent of #311 (seed forwarding) and #309 (single-score formatting of zero intervals).
Validation on macOS, Python 3.12.14, NumPy 2.5.4:
python -m pytest --ignore=test/test_api.py --ignore=test/test_dataset.py: 183 passed, including the existing WMT17 significance comparisons.mypy sacrebleu scripts test: passes all 45 files; Ruff passes on both changed files.19.5and CI half-width18.5for both float32 and float64 input arrays.Dataset-download/API network tests and the network-heavy shell suite were not completed locally. Whole-repository Ruff reports the existing circular-import
E402atsacrebleu/utils.py:650, which is unchanged.AI assistance: an automated AI assistant performed the investigation, implementation, tests, and description. No independent human review has occurred.