Repository navigation
build01: stop calling an omission permitted that §1.4 never permitted - #18
Merged
Merged
Conversation
hongdy22
force-pushed
the
fix/optional-check-semantics
branch
from
September 14, 2026 05:46
0943df9 to
bbbb4f6
Compare
`srb_skip_ok` decides how a skip is recorded, not whether it is charged, and `pytest_module`'s docstring draws the line it was narrowed to: where the reference has no answer the check is not a question and is deleted from its suite; where the reference does have one, a submission that cannot be asked has failed to produce the thing being asked about, and the submissions that did produce it should not be the only ones paying for the check. `test_optional_macro_value_when_present` sat on the wrong side of that line. State A's configure defines all 25 of these macros -- they are where the recorded values in macros.json come from -- and across the preserved build01 cells the row reads 189 pass and 40 fail beside its skips. It is a question submissions answer. §1.4 asks for the same macro set and names no exception, and nothing else in the instruction mentions optional, permitted or skip. So the marker goes and the reason stops saying "(permitted)". The machinery is unchanged: absence still raises `pytest.skip`, and without the marker the harness records it as a failure saying the contract does not permit it. The `OPTIONAL` name stays, and a comment says what the group actually is -- 25 macros graded by value in one check rather than by presence in a second, which is the only way they differ from the 64 above. This reverses the earlier direction on this branch, which recorded an absent macro as a pass. That was a judgement about autoconf bookkeeping substituted for the rule the tree already had, and the corpus settles it the other way.
hongdy22
force-pushed
the
fix/optional-check-semantics
branch
from
September 14, 2026 06:19
bbbb4f6 to
63d97fd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #10.
Rewritten. The first version recorded an absent macro as a pass, and also changed fw02. Both are reverted — the scoring was never wrong.
skipis how every check in this tree records that the thing being asked about was not produced. It is charged, it does not pass, andsrb_skip_okonly decides the wording. That is the published rule, inscores_raw/LEGEND.md:It is applied that way with no exception. Across the published tables, 313 skipped checks on 48 module rows fall on 42 cells, and all 42 scored 0.
These 25 macros are required, on all three counts that decide it:
instruction.mdcontains no optional, permitted, may omit or skip;configuredefines all 25, which is where the recorded values indata/macros.jsoncome from;So no published score changes. What was wrong was the suite calling the omission
(permitted)and carryingsrb_skip_ok, which told a submission an omission was free when it was not. Both are gone; absence is now recorded as a failure saying the contract does not permit it, and a wrong value fails as before.Regression:
infra/tests/test_build01_macro_contract.py, 15 checks, fails onmain(2 of 15). The seven infra suites stay at 556 passed.test_ssp_can_be_turned_offhad the same wording and is fixed in #17, which already edits that function.