fix: MC/DC gate no longer silently passes on empty parse; branch-proxy honestly labeled - #117
Merged
Merged
Conversation
…y is honestly labeled (issue found in quality review)
Two real defects found while assessing the quality of c-FuSa's MC/DC
support, both fixed:
1. parse_mcdc_json() treated 'zero condition records found' as a
PASS ('nothing to fail'). That's indistinguishable from a
wrong/empty/malformed --mcdc-file purely from content -- a bad
path, a truncated file, or an LLVM export-format change the
naive string-scan no longer recognizes would all silently read as
'MC/DC verified' instead of erroring. Same failure shape as the
MAX_REQS silent-truncation bug (issue #100) fixed earlier: an
asserted --mcdc-file must produce a real answer, not a pass that
could just as easily mean nothing was parsed. Now fails loudly
with a diagnostic note; coverage_pct also no longer shows a
contradictory '100.00% (0/0 conditions) FAIL'.
2. Without --mcdc-file, the DAL-A/ASIL-D-required MC/DC gate falls
back to treating 100% branch coverage as a proxy -- but the tool
labeled this output 'MC/DC analysis', which is materially
misleading: 100% branch/decision coverage does not establish that
every condition within a decision independently affects its
outcome (the entire reason MC/DC exists as a distinct, stricter
metric). A DAL-A/ASIL-D project relying on the fallback would see
'MC/DC analysis: PASS' without having demonstrated real MC/DC.
Now: a stderr WARNING when the proxy is used, the text-mode label
changed to 'MC/DC gate (branch-coverage proxy -- NOT verified
MC/DC)' with an explanatory note, a machine-readable
'mcdcProxy': {verified: false, ...} JSON field, and --help text
spelling out the distinction.
Updated the one existing test that asserted the old silent-pass
behavior (renamed test_mcdc_no_records_passes ->
test_mcdc_no_records_fails, now asserts rc=1). Added 3 new
regression tests: a garbage (non-MC/DC-export) file also fails, and
the failure note/JSON both explain why rather than leaving a bare
exit code.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Matt Jones <47545907+SoundMatt@users.noreply.github.com>
| char buf[4096] = ""; | ||
| size_t n = fread(buf, 1, sizeof(buf) - 1, f); | ||
| buf[n] = '\0'; | ||
| fclose(f); |
| snprintf(path, sizeof(path), "%s/no_records2.json", MCDC_TEST_DIR); | ||
| char out[256]; | ||
| snprintf(out, sizeof(out), "%s/no_records_out.json", MCDC_TEST_DIR); | ||
| char *argv[] = {"cfusa", "coverage", "--mcdc-file", path, |
SoundMatt
added a commit
that referenced
this pull request
Aug 14, 2026
Covers the MC/DC gate honesty fixes merged in PR #117. Signed-off-by: Matt Jones <47545907+SoundMatt@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two real defects found while assessing the quality of c-FuSa's MC/DC support (asked directly, not filed as a separate issue first — fixing on request).
1. Silent pass on empty/malformed
--mcdc-fileparse_mcdc_json()treated "zero condition records found" as a PASS ("nothing to fail"). That's indistinguishable from a wrong/empty/malformed--mcdc-filepurely from content — a bad path, a truncated file, or a future LLVM export-format change the naive string-scan no longer recognizes would all silently read as "MC/DC verified" instead of erroring.This is the exact same failure shape as the
MAX_REQSsilent-truncation bug from earlier today (#100): an asserted--mcdc-filemust produce a real answer, not a pass that could just as easily mean nothing was parsed. Now fails loudly with a diagnostic note. Also fixed a cosmetic-but-confusing side effect:coverage_pctstayed at its100.0default in this path, so the output used to show a contradictory"100.00% (0/0 conditions) FAIL"— now0.00%.2. The branch-coverage fallback was labeled "MC/DC"
Without
--mcdc-file, the DAL-A/ASIL-D-required MC/DC gate falls back to treating 100% branch coverage as a proxy — but the tool labeled this output "MC/DC analysis", which is materially misleading. 100% branch/decision coverage does not establish that every condition within a decision independently affects its outcome — the entire reason MC/DC exists as a distinct, stricter metric (if (a && b && c)hits 100% branch coverage with 2 test vectors; MC/DC needs enough to isolate each condition). A DAL-A/ASIL-D project relying on the fallback would see "MC/DC analysis: PASS" without having demonstrated real MC/DC.Now:
WARNINGprinted whenever the proxy is used.MC/DC gate (branch-coverage proxy — NOT verified MC/DC)with an explanatory note."mcdcProxy": {"verified": false, ...}JSON field (so a consumer parsing the report for certification evidence can't mistake it for a realmcdcReport).--helptext spells out the distinction explicitly.Testing
test_mcdc_no_records_passes→test_mcdc_no_records_fails, now assertsrc=1).ctest: 42/42 passing, including the pre-existingtest_coverage_asilsuite from cfusa coverage: add an ASIL-aware MC/DC gate (currently DO-178C --dal only) #106 (confirms the branch-proxy PASS/FAIL logic itself is unchanged — only labeling and the empty-parse case changed).cfusa check --dir .: 0 errors.cfusa trace --dir .: newREQ-COV021traced + tested, no dangling references.🤖 Generated with Claude Code