Skip to content

test: pass without a network or origin/v8 - #4894

Open
bercedev wants to merge 1 commit into
Graphify-Labs:v8from
bercedev:test/no-network-or-origin-v8
Open

bercedev wants to merge 1 commit into
Graphify-Labs:v8from
bercedev:test/no-network-or-origin-v8

Conversation

@bercedev

Copy link
Copy Markdown
Contributor

What does this PR do?

On a clean v8, uv run pytest tests has 18 failures when the checkout has no network or no origin/v8, for example a fork where origin isn't this repo. CONTRIBUTING says tests must not depend on the contributor's environment, so this makes them not depend on it.

  • 7 in test_security.py: validate_url resolves the host, so example.com and arxiv.org needed real DNS and failed with ValueError: DNS resolution failed. A new public_dns fixture fakes socket.getaddrinfo the way test_ollama.py already does. It's only on those 7, the SSRF tests aren't touched.
  • 11 in test_skillgen.py: they read blobs from origin/v8 and died with SystemExit: error: could not read origin/v8:.... They now skip with needs_origin_v8, which uses the same gen._v8_available() probe the CLI validators already skip on.

Nothing changes under graphify/ or tools/.

Limitation: if CI ever lost fetch-depth: 0, these 11 would skip and not fail. The CLI validators in skillgen-check already behave that way.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Tests or CI
  • Refactor
  • Security fix

Verification & Invariants

Invariant: a test result doesn't depend on the network or on local refs. With origin/v8 present nothing gets skipped, so the audits still guard what they did.

  • Read the CONTRIBUTING.md guide.
  • Reproduced the issue and identified the invariant.
  • Made the smallest fix necessary.
  • Added a regression test (if bug fix) or isolated boundary test.
  • Kept the PR description synchronized with the final implementation.
  • Documented any limitations / unsupported cases explicitly.

How was this tested?

--network=none, no .git
  tests/test_security.py tests/test_skillgen.py    140 passed, 11 skipped
  full suite                                       8939 passed, 44 skipped, 1 xfailed, 0 failed
  full suite on a clean v8                         18 failed (the ones above)
--network=none, with .git and origin/v8
  tests/test_security.py tests/test_skillgen.py    151 passed, 0 skipped
ruff check .                                       clean

Graphify-specific checklist

  • I updated generated skill artifacts (uv run python -m tools.skillgen --bless) when changing their source fragments.
  • I confirmed that AST/structural extraction remains deterministic (no ambient state dependencies like ENV variables).
  • I reviewed changes for security implications (no unsafe interpolation into shell/Python).
  • I confirmed no API keys or local-only graph data are included.
  • (If applicable) I disclosed AI authorship in my commit messages.

Note: I used Claude Code to write this and ran the tests above locally.

7 security tests needed real DNS through validate_url, and 11 skillgen tests
read blobs from origin/v8 and died when the checkout didn't have it.
@bercedev
bercedev requested a review from safishamsi as a code owner October 30, 2026 21:47
@github-actions

Copy link
Copy Markdown

Thanks for the pull request, @bercedev. A maintainer will review it soon.

Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions.

A couple of things that speed up review: make sure the test suite passes on Python 3.10 and 3.13, and that the change keeps extraction deterministic.

@github-actions github-actions Bot added the contributor Pull request from a returning community contributor label Oct 30, 2026

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.

Not checked: tests were not run; no formal proof of the changed code.


Graphify review — findings

Stops two test suites from depending on the environment: the URL/fetch tests no longer need a network, and the skillgen audit/roundtrip tests skip cleanly when origin/v8 isn't in the checkout. - test_security.py: a new public_dns fixture stubs socket.getaddrinfo to return a public IP. The validate_url and safe_fetch* tests that reach the host-resolution step use it, so they no longer do real DNS lookups. - test_skillgen.py: a needs_origin_v8 marker, backed by gen._v8_available(), skips every test that reads v8 blobs (audit-coverage, roundtrip, sanctioned-lines, always-on byte-faithfulness) on forks or tarball checkouts. CI uses fetch-depth: 0, so they still run there.

No blocking issues surfaced.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 232 functions depend on the 232 functions this change touches.

Health — grade A; 1 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):

  • test_audit_catches_a_dropped_non_allowlisted_heading() — 0 callers, 6 callees (medium)

Verification — 232 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 232 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

2 of 469 test file(s) selected (0%) via static blast radius.

  • tests/test_security.py — impact, changed-test
  • tests/test_skillgen.py — impact, changed-test

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor Pull request from a returning community contributor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant