Skip to content

feat(report): annotate static findings with structured source roles - #811

Open
optimization2026 wants to merge 1 commit into
NVIDIA:mainfrom
AISOP-dev:feat/structured-source-roles
Open

optimization2026 wants to merge 1 commit into
NVIDIA:mainfrom
AISOP-dev:feat/structured-source-roles

Conversation

@optimization2026

Copy link
Copy Markdown

Summary

Add bounded, report-only source-role annotations for static findings in supported local AISOP/AISP JSON files.

This is a conservative first increment of Phase 2 in #130, building on the phase-1 discovery and summaries in #211. It does not implement Phase 3 or close the broader issue.

The practical improvement is occurrence-level context: when identical text appears in an executable step and a constraint, each reported occurrence receives its own structural role and JSON Pointer rather than borrowing the representative occurrence's context.

Implementation

  • Index admitted, cached UTF-8 source bytes with bounded JSON value spans. Reject decoded duplicate keys, invalid input and unsupported layouts rather than assigning an exact role.
  • Attach evidence.structured_source_role to occurrence-specific display copies, after baseline suppression, risk calculation and finding compaction.
  • Include the original character span, cached-content digest, structural role, mapping status and a bounded reason. risk_polarity remains unknown; role_confidence is null, not an uncalibrated probability.
  • Reuse existing terminal, Markdown, JSON and SARIF sanitization/rendering. Keep phase-1 summaries separate and expose auxiliary mapping coverage.
  • Bound documents, cached bytes, annotation records, pointer length, mapping time and ASCII-escaped annotation JSON size. Exhausting an annotation budget omits auxiliary context, not the original finding.

No new production dependency, workflow execution, resource traversal or filesystem reopening is introduced.

Safety and scope boundaries

Canonical findings, fingerprints, original severities, detection confidence, suppression decisions and risk results are not enriched or downgraded. The MCP response's embedded report carries annotations; its canonical findings array remains unchanged.

Precise mapping requires an admitted local .aisop.json source, matching raw/text caches, scanner-owned static origin and exact original columns. Transitive provenance, normalized or reconstructed views, missing columns and ambiguous ownership remain unknown. Suppressed findings are not annotated.

A structural constraint label is not proof that a policy was enforced or that the text is benign. Non-reserved function fields remain executable steps even when named hard_deny or example.

This is source-location context, not protocol certification, runtime conformance or a demonstrated reduction in false positives.

Validation

Submitted change on the updated base

Windows, Python 3.12.9, with an independent installed environment and verified imports from the submission checkout.

Base: a0d489e67f847aeedb5e6d471adfbbafc9e15670.

  • The three added feature-test files passed.
  • Seven selected upstream regression files: 500 passed, 1 skipped. The skip requires POSIX FIFO support.
  • Ruff lint and format checks passed for src/ and tests/.
  • Staged whitespace checks passed.
  • The eight feature-file hashes matched the reviewed candidate after moving to this base.
  • No local Windows test-compatibility changes were applied to the submission checkout or included in this PR.

The selected regression files cover reporting, deduplication, suppression, finding models, phase-1 structured summaries, shared source-location helpers and static-runner filtering.

Commands for the selected test groups
python -m pytest -q -ra -m "not integration and not provider" \
  tests/unit/test_structured_source.py \
  tests/unit/test_structured_role_report.py \
  tests/nodes/test_structured_role_reporting.py
python -m pytest -q -ra -m "not integration and not provider" \
  tests/nodes/test_report.py \
  tests/nodes/test_deduplicate.py \
  tests/unit/test_suppression.py \
  tests/test_models.py \
  tests/nodes/analyzers/test_structured_skill_roles.py \
  tests/nodes/analyzers/test_common.py \
  tests/nodes/analyzers/test_static_runner_filtering.py
ruff check --no-fix src/ tests/
ruff format --check src/ tests/

These are targeted checks, not the complete upstream CI suite. The existing test_static_runner_prepared_context.py was not included in this clean-checkout selection.

Earlier evidence, kept separate by revision

On the original base, 3c8e4b958fda9602e0044885f766be0e49f03585, the same final eight feature files passed 6/6 separate static-only native-directory acceptance cases: graph invocation, the in-process CLI application and MCP run_scan, each with JSON and SARIF output.

The synthetic directory had no SKILL.md sidecar. The checks verified both P1 occurrence roles and original source ranges, source digests, report structure, and canonical-finding isolation where exposed. This did not test the console launcher or MCP transport. These six cases were not rerun after the base update.

An earlier, pre-budget-fix candidate completed a broader regular Windows run with 10,693 passed and 16 failed. Eleven non-release failures were reproduced on the clean original baseline with matching classified outcomes. Five release-test failures remain unvalidated under safe Windows fake-CLI isolation. These failures are not treated as passes.

The complete upstream CI suite has not passed locally, and no current-candidate full-suite or complete cross-platform result is claimed.

Change boundary

Eight files only: two existing production files, two new production modules, three test files and one documentation file.

No changes to dependency manifests, lockfiles, release tooling, CI workflows or existing upstream tests. Local diagnostic scripts and test-compatibility patches are not part of this PR.

Refs #130. Follow-up to #211.

Add bounded, occurrence-level source-role annotations to reports while preserving existing findings and risk scoring.

Refs NVIDIA#130. Follow-up to NVIDIA#211.

Signed-off-by: AISOP.dev <noreply@AISOP.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant