Skip to content

static_parse_limit on valid Rust, Python and POSIX shell source leaves files partially inspected #694

Description

@elliottwaves-20

Summary

The bounded shell parser behind has_bounded_parse_exhaustion (static_patterns_tool_misuse) reports static_parse_limit for valid source files that contain no shell at all, or only an ordinary shell idiom. The file then counts as partially inspected, so the target can never reach safe_to_install, whatever else the scan finds. This looks related to #628 and #515 but still reproduces on current main.

Versions: v2.12.0 (c7958a3) and main at 2226747, --no-llm, Windows 11 / Python 3.12.

Reproducers

1. POSIX shell, 5 lines (install.sh, the idiom rustup-init.sh uses at line 246 to detect ELF files):

#!/bin/sh
_current_exe_head=$(head -c 5 "$1")
if [ "$_current_exe_head" = "$(printf '\177ELF\001')" ]; then
    echo elf
fi

2. Rust: src/lib.rs of the itoa 1.0.18 crate (crates.io). The trace ends at line 248, a comment:

// SAFETY: `offset` is always included between 0 and `buf`'s length.

3. Python: a literal backtick inside a regex character class. The generator below writes an ~8 KB file; with ~4 KB it still passes, so the unmatched backtick seems to be treated as a command substitution that runs to the end of the file:

import pathlib
T, BS = chr(96), chr(92)
head = ("import re\n\nNET_TOOLS = (\"curl\", \"wget\", \"nc\")\n"
        "NET_TOOL_WORD = re.compile(r\"(?:^|[" + BS + "s;&|" + T + "(/])(\" + \"|\".join(NET_TOOLS) + r\")(?=" + BS + "s|$)\")\n")
body = "".join(f"\n\ndef check_{i}(argv):\n    \"\"\"Return the network tool named in argument {i}, if any.\"\"\"\n"
               f"    match = NET_TOOL_WORD.search(argv[{i}] if len(argv) > {i} else \"\")\n"
               f"    return match.group(1) if match else None\n" for i in range(40))
pathlib.Path("repro/net.py").parent.mkdir(exist_ok=True)
pathlib.Path("repro/net.py").write_text(head + body, encoding="utf-8")

Each case: skillspector scan <dir> --no-llm --format json --output r.json

Actual

"partially_inspected_files": 1,
"ledger_exceptions": [{"reason_code": "static_parse_limit", "phase": "static", ...}]

Expected

These files complete static analysis. For non-shell languages (Rust, Python), backticks and apostrophes in comments, regexes and string literals are not shell syntax. For case 1, $(printf '\177ELF\001') is a bounded, literal command substitution.

Found while scanning well-known packages and our own security tooling with SkillSpector before installation; happy to test a fix.

Activity

  1. jamesjsanders commented on Oct 3, 2026

    @jamesjsanders

    Two related shell cases, checked on v2.12.0 and main at 3527006 (--no-llm):

    1. A destructive candidate found inside a double-quoted string has its operands tokenized in top-level context, so the string's closing quote opens a new quote and the file is reported static_parse_limit. The rm finding itself is correct; only completeness is wrong.

       CMD="sudo rm -rf /"
       $CMD
      
    2. Backtick substitution inside double quotes is not a destructive candidate: _destructive_command_words('echo "`rm -rf /`"\n') yields nothing, while the $(rm -rf /) form yields rm.

  2. rng1995 commented on Oct 4, 2026

    @rng1995
    Collaborator

    The open PR #704 addresses the provable POSIX printf-escape reproduction only. It has not merged and does not claim to resolve the Rust/Python ownership cases or the subsequently reported quoted-command/backtick cases. Keeping this broader parser report open for the remaining scope.

    PR-state snapshot checked on 2026-10-04:

    • PR #704 — open, non-draft; no aggregate review decision reported; latest reported check rollup: success.

    PR #704 covers only the provable POSIX printf-escape case; Rust/Python and subsequent quoted-command/backtick cases remain outstanding.

    Keeping this issue open: the relevant implementation is not merged and the remaining scope still needs verification. Check/review status is a point-in-time snapshot, not a claim of merge readiness.

  3. njfwzc4m96-arch commented on Oct 5, 2026

    @njfwzc4m96-arch

    We can also reproduce this host-language parsing problem in ordinary JavaScript template literals on macOS / Python 3.12, using official main at 4a55062 (package version 2.12.0).

    The following synthetic test only analyzes strings; it does not execute the JavaScript or shell text:

    from skillspector.nodes.analyzers import static_patterns_tool_misuse as tm
    padding = '\n// ordinary padding\n' * 400
    cases = {
        'filename': "const output = path.resolve(outDir, `${id}_${String(frame).padStart(5, '0')}.png`);",
        'random_seed': 'const r = (s) => random(`${startFrame}-${i}-${s}`);',
        'svg_points': 'pts.push(`${cx + Math.cos(t) * rx} ${cy + Math.sin(t) * ry}`);',
        'scene_error': 'throw new Error(`Unknown scene kind "${kind}". Known: ${Object.keys(kinds).join(", ")}`);',
        'dynamic_command_control': 'exec(`${command} -rf /`);',
    }
    for name, source in cases.items():
        print(name, tm.has_bounded_parse_exhaustion(
            source + padding, lambda: None, file_type='javascript'))

    Observed on that official revision: True for all five cases. The first four are filename/seed/geometry/message construction, while the last must retain a warning or incomplete outcome. Full-file context matters, so shorter isolated strings do not always reproduce every host-source failure.

    We need a maintainer-supported correction that recognizes valid host syntax while preserving embedded commands, dynamic-command uncertainty, malformed-source fail-closed behavior, and parser resource bounds. We are not asking to disable analysis, suppress findings, change scores, or treat incomplete coverage as complete.

    We see that PR #704 is still open and explicitly covers the POSIX printf case only. Is there an approved fix or development branch addressing this host-language case, or would maintainers accept a focused contribution with paired benign/adversarial regressions? Please identify the official release or supported commit to validate against. We can test it and provide sanitized results.

    Separately, the exact JSON below receives TM1/HIGH for its defensive command prohibition:

    {"permissions":{"deny":["Bash(rm -rf *)"]}}

    Would you prefer a separate false-positive issue for recognizing genuine permission-denial context while retaining checks on active execution and misleading policy-like text?

    No proprietary skill source, private paths, user identity, credentials, full reports, or unpublished vulnerability details are included here. No clean-installation status is claimed. This request concerns reproducible false positives/incomplete checks, not a confirmed vulnerability in an official release.

  4. elliottwaves-20 commented on Oct 5, 2026

    @elliottwaves-20
    Author

    Three more host-syntax cases from the same family, found while triaging AE1 findings on a set of public skills. Checked on main at 532f67c and on the PR #704 head (9044d94); all three still return True on both. Like the reproduction above, the snippet only analyzes strings and executes nothing.

    from skillspector.nodes.analyzers import static_patterns_tool_misuse as tm
    
    padding = "\n# ordinary padding\n" * 400
    cases = {
        # Markdown table cell that names a parameter
        "markdown_table_cell": ("markdown", "|---|---|\n| timeout | Max wait in seconds |\n"),
        # Apostrophe in a Python comment
        "python_comment": ("python", "# Prefix doesn't include the array name\n" + padding),
        # PowerShell subexpression inside a double-quoted string
        "powershell_subexpression": ("powershell", 'Write-Warning "Stale path: $($cfg.command)"\n' + padding),
    }
    for name, (file_type, source) in cases.items():
        print(name, tm.has_bounded_parse_exhaustion(
            source, lambda: None, file_type=file_type, complete_context=True))

    Observations:

    1. Markdown table cells. A cell whose first word is a command wrapper is parsed as a command, and the following | exhausts the span. timeout, sudo, nice and xargs reproduce. env, nohup, time, exec and command do not. Ordinary parameter tables in API reference docs (| timeout | float | 150 | ... |) are the common trigger. It reproduces with no padding at all.
    2. Python comments. A single apostrophe in a # comment (doesn't) is enough once the file has some length. The same line without the apostrophe is clean.
    3. PowerShell $(...) in double-quoted strings. "... $($obj.prop)" is ordinary string interpolation. A single-quoted variable in the same string ('$n') on its own does not trigger it.

    Impact on our side: in a static scan of 24 public skills with open AE1 findings, 37 of the 38 affected target files had static_parse_limit as the only reason. The files were read in full; only this parser's span was left incomplete. On main, AE1 disappeared for 3 of the 24 skills, which shows that #627/#634/#686 help. We localized nine of the remaining triggers. Most fall into the three cases above or the JavaScript template literals already reported here. One is a genuinely complex shell pipeline (curl … | sudo tar …), and we expect that one to stay partial.

    Like the previous commenter, we are not asking to suppress findings or to treat partial coverage as complete. If maintainers would accept it, I can open a focused PR for the Markdown table-cell case and the Python comment case. It would come with paired regressions: benign cells and comments stay complete, while real commands in a table cell or after the comment keep their findings and their partial outcome. Please let me know whether this would overlap with work already planned for the JavaScript case.

  5. rng1995 commented on Oct 6, 2026

    @rng1995
    Collaborator

    Merge-state update checked on 2026-10-06: PR #704 has merged and addresses the provable POSIX printf-escape case.

    Keeping this issue open for the remaining host-syntax ownership cases: Rust/Python, quoted-command/backtick handling, JavaScript template literals, Markdown table cells, Python comment apostrophes, and PowerShell interpolation. PR #740 bounds separate XOR/function-header parsing; it does not resolve those cases. The earlier Markdown/parser fixes also do not establish that all of these files are completely inspected.

    No blanket suppression or conversion of partial coverage to complete is implied by the merged fix.

  6. efegokdemir commented on Oct 7, 2026

    @efegokdemir
    Contributor

    I’m preparing a focused JavaScript template-literal ownership fix for #694. The change will use parsed host syntax to keep ordinary templates from exhausting the shell parser, while preserving partial coverage for dynamic templates passed to command-execution APIs. I’ll include paired regression tests.

  7. rng1995 commented on Oct 8, 2026

    @rng1995
    Collaborator

    Merge-state update checked on 2026-10-08: PR #778 has merged the validated Python string/comment ownership and narrow Perl eval-block handling. Related marker-parser fixes have also merged: #777 for comparison operators mistaken for tag openers, and #784 for proven Python literal closing quotes. These add to the POSIX printf fix in #704.

    Keeping this broader issue open. Remaining cases include Rust, JavaScript template literals, Markdown wrapper/table cells, PowerShell/quoted expansions, and quoted-command/backtick handling. Active partial implementations are #779, #780, and #801; all remain unmerged.

    The Python fixes are deliberately bounded to proven host syntax. They are not blanket exemptions for all Python strings, malformed source, dynamic execution, or parser exhaustion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions