Baseline generation silently drops findings: dedupe key is coarser than match scope
Version: skillspector 2.12.0, static-only (--no-llm).
Repro: scan a skill tree reporting 47 findings, run skillspector baseline -o .skillspector-baseline.yaml, rescan with --baseline. Result is deterministic across runs: exactly 27 suppressed, 20 remain — and the 20 are all previously-reported instances, not new findings.
Root cause (one mechanism, two symptoms): generation dedupes by (rule_id, match-content) across files (the emitted file holds exactly one entry per distinct content: 10 distinct AE1 artifacts → 10 entries; rules: []), while suppression matching is occurrence/file-scoped. Consequences:
- Repeat occurrences never match. 10 AE1 re-references, 3 TM1 repeats, 1 RP1 near-duplicate block — same content already has an entry for the file, yet the repeats still fire.
- Worse: whole file groups get zero entries (silent drop). Six identical
keyring PE3 hits in scripts/check-prerequisites.sh produced no baseline entry, while the single content-identical PE3 in references/credentials-setup.md (same match_fingerprint 84a991…) got one. Same for one TM1 in scripts/responsive-screenshots.sh vs its twin in scripts/quick-debug.sh. The generator collapsed cross-file duplicates into a single entry, then file-scoped matching suppressed only that file — the other files were left with nothing, silently.
Expected: either the dedupe key includes the file (one entry per occurrence-scoped match), or matching treats content-identical cross-file hits as covered. At minimum, baseline should warn when reported findings cannot be fingerprinted distinctly, instead of writing a file that claims coverage it can't deliver.
Happy to provide the two scan JSONs (47-report and 27-suppressed/20-remaining report) if useful.
Baseline generation silently drops findings: dedupe key is coarser than match scope
Version: skillspector 2.12.0, static-only (
--no-llm).Repro: scan a skill tree reporting 47 findings, run
skillspector baseline -o .skillspector-baseline.yaml, rescan with--baseline. Result is deterministic across runs: exactly 27 suppressed, 20 remain — and the 20 are all previously-reported instances, not new findings.Root cause (one mechanism, two symptoms): generation dedupes by
(rule_id, match-content)across files (the emitted file holds exactly one entry per distinct content: 10 distinct AE1 artifacts → 10 entries;rules: []), while suppression matching is occurrence/file-scoped. Consequences:keyringPE3 hits inscripts/check-prerequisites.shproduced no baseline entry, while the single content-identical PE3 inreferences/credentials-setup.md(samematch_fingerprint84a991…) got one. Same for one TM1 inscripts/responsive-screenshots.shvs its twin inscripts/quick-debug.sh. The generator collapsed cross-file duplicates into a single entry, then file-scoped matching suppressed only that file — the other files were left with nothing, silently.Expected: either the dedupe key includes the file (one entry per occurrence-scoped match), or matching treats content-identical cross-file hits as covered. At minimum,
baselineshould warn when reported findings cannot be fingerprinted distinctly, instead of writing a file that claims coverage it can't deliver.Happy to provide the two scan JSONs (47-report and 27-suppressed/20-remaining report) if useful.