Skip to content

fix: MC/DC gate no longer silently passes on empty parse; branch-proxy honestly labeled - #117

Merged
SoundMatt merged 1 commit into
mainfrom
fix/mcdc-honesty-and-silent-pass
Aug 14, 2026
Merged

fix: MC/DC gate no longer silently passes on empty parse; branch-proxy honestly labeled#117
SoundMatt merged 1 commit into
mainfrom
fix/mcdc-honesty-and-silent-pass

Conversation

@SoundMatt

Copy link
Copy Markdown
Owner

Two real defects found while assessing the quality of c-FuSa's MC/DC support (asked directly, not filed as a separate issue first — fixing on request).

1. Silent pass on empty/malformed --mcdc-file

parse_mcdc_json() treated "zero condition records found" as a PASS ("nothing to fail"). That's indistinguishable from a wrong/empty/malformed --mcdc-file purely from content — a bad path, a truncated file, or a future LLVM export-format change the naive string-scan no longer recognizes would all silently read as "MC/DC verified" instead of erroring.

This is the exact same failure shape as the MAX_REQS silent-truncation bug from earlier today (#100): an asserted --mcdc-file must produce a real answer, not a pass that could just as easily mean nothing was parsed. Now fails loudly with a diagnostic note. Also fixed a cosmetic-but-confusing side effect: coverage_pct stayed at its 100.0 default in this path, so the output used to show a contradictory "100.00% (0/0 conditions) FAIL" — now 0.00%.

2. The branch-coverage fallback was labeled "MC/DC"

Without --mcdc-file, the DAL-A/ASIL-D-required MC/DC gate falls back to treating 100% branch coverage as a proxy — but the tool labeled this output "MC/DC analysis", which is materially misleading. 100% branch/decision coverage does not establish that every condition within a decision independently affects its outcome — the entire reason MC/DC exists as a distinct, stricter metric (if (a && b && c) hits 100% branch coverage with 2 test vectors; MC/DC needs enough to isolate each condition). A DAL-A/ASIL-D project relying on the fallback would see "MC/DC analysis: PASS" without having demonstrated real MC/DC.

Now:

  • A stderr WARNING printed whenever the proxy is used.
  • Text-mode label changed to MC/DC gate (branch-coverage proxy — NOT verified MC/DC) with an explanatory note.
  • Machine-readable "mcdcProxy": {"verified": false, ...} JSON field (so a consumer parsing the report for certification evidence can't mistake it for a real mcdcReport).
  • --help text spells out the distinction explicitly.
$ cfusa coverage --lcov coverage.info --asil ASIL-D
cfusa coverage: WARNING: MC/DC is required at this DAL/ASIL level, but no
--mcdc-file was given — falling back to 100% branch coverage as a proxy.
Branch/decision coverage does NOT establish MC/DC coverage; this is not
verified MC/DC evidence. Provide --mcdc-file <llvm-mcdc.json> ...
...
  MC/DC gate (branch-coverage proxy — NOT verified MC/DC): 100.00%
  NOTE: no --mcdc-file was given; this result is NOT verified MC/DC
  evidence — provide --mcdc-file <llvm-mcdc.json> for a real MC/DC gate.

Testing

  • Updated the one existing test that asserted the old silent-pass behavior (test_mcdc_no_records_passestest_mcdc_no_records_fails, now asserts rc=1).
  • 3 new regression tests: a garbage (non-MC/DC-export) file also fails; the failure note and JSON output both explain why, not just a bare exit code.
  • Manually verified the proxy warning/label/JSON field all fire correctly against a real build (shown above).
  • ctest: 42/42 passing, including the pre-existing test_coverage_asil suite from cfusa coverage: add an ASIL-aware MC/DC gate (currently DO-178C --dal only) #106 (confirms the branch-proxy PASS/FAIL logic itself is unchanged — only labeling and the empty-parse case changed).
  • cfusa check --dir .: 0 errors.
  • cfusa trace --dir .: new REQ-COV021 traced + tested, no dangling references.

🤖 Generated with Claude Code

…y is honestly labeled (issue found in quality review)

Two real defects found while assessing the quality of c-FuSa's MC/DC
support, both fixed:

1. parse_mcdc_json() treated 'zero condition records found' as a
   PASS ('nothing to fail'). That's indistinguishable from a
   wrong/empty/malformed --mcdc-file purely from content -- a bad
   path, a truncated file, or an LLVM export-format change the
   naive string-scan no longer recognizes would all silently read as
   'MC/DC verified' instead of erroring. Same failure shape as the
   MAX_REQS silent-truncation bug (issue #100) fixed earlier: an
   asserted --mcdc-file must produce a real answer, not a pass that
   could just as easily mean nothing was parsed. Now fails loudly
   with a diagnostic note; coverage_pct also no longer shows a
   contradictory '100.00% (0/0 conditions) FAIL'.

2. Without --mcdc-file, the DAL-A/ASIL-D-required MC/DC gate falls
   back to treating 100% branch coverage as a proxy -- but the tool
   labeled this output 'MC/DC analysis', which is materially
   misleading: 100% branch/decision coverage does not establish that
   every condition within a decision independently affects its
   outcome (the entire reason MC/DC exists as a distinct, stricter
   metric). A DAL-A/ASIL-D project relying on the fallback would see
   'MC/DC analysis: PASS' without having demonstrated real MC/DC.
   Now: a stderr WARNING when the proxy is used, the text-mode label
   changed to 'MC/DC gate (branch-coverage proxy -- NOT verified
   MC/DC)' with an explanatory note, a machine-readable
   'mcdcProxy': {verified: false, ...} JSON field, and --help text
   spelling out the distinction.

Updated the one existing test that asserted the old silent-pass
behavior (renamed test_mcdc_no_records_passes ->
test_mcdc_no_records_fails, now asserts rc=1). Added 3 new
regression tests: a garbage (non-MC/DC-export) file also fails, and
the failure note/JSON both explain why rather than leaving a bare
exit code.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Signed-off-by: Matt Jones <47545907+SoundMatt@users.noreply.github.com>
Comment thread tests/test_mcdc.c
char buf[4096] = "";
size_t n = fread(buf, 1, sizeof(buf) - 1, f);
buf[n] = '\0';
fclose(f);
Comment thread tests/test_mcdc.c
snprintf(path, sizeof(path), "%s/no_records2.json", MCDC_TEST_DIR);
char out[256];
snprintf(out, sizeof(out), "%s/no_records_out.json", MCDC_TEST_DIR);
char *argv[] = {"cfusa", "coverage", "--mcdc-file", path,
@SoundMatt
SoundMatt merged commit b3403bc into main Aug 14, 2026
10 checks passed
SoundMatt added a commit that referenced this pull request Aug 14, 2026
Covers the MC/DC gate honesty fixes merged in PR #117.

Signed-off-by: Matt Jones <47545907+SoundMatt@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants