[rahul05ranjan]

What does this PR do?

An oversized splittable file is dispatched as several FileSlice chunks (expand_oversized_files → per-chunk _checkpoint_chunk saves). If the run is interrupted before every slice's chunk returns (crash, kill, or a KeyboardInterrupt-style abort), the checkpoint written by the slices that did finish is stamped clean, so the parent file sits in the semantic cache looking complete. check_semantic_cache then reports zero uncached files and the content that was never extracted (everything past the completed slice) is silently omitted from every later graph.

Fixes #4943. _checkpoint_chunk now stamps a sliced file partial: True whenever this chunk does not contain all of the file's slices (FileSlice.index/total), merged with the existing _partial_source_files truncation set. Interrupted runs leave a partial entry that load_cached/check_semantic_cache treat as a miss, so the whole file re-dispatches next run. A fully completed run is unaffected: checkpoints stay conservative (same contract an LLM-truncated slice already has), and the final full save of a completed run — the non-merge save_semantic_cache call the CLI makes after the corpus run — overwrites the stamp with a clean complete entry.

Type of change

• Bug fix
• Tests or CI

Verification & Invariants

Invariant: a semantic cache entry for a file may only be served as complete when every slice of that file completed; otherwise the entry must carry partial: True.

• Read the CONTRIBUTING.md guide.
• Reproduced the issue and identified the invariant.
• Made the smallest fix necessary (one call site, no schema or cache-format change — the partial flag machinery already existed for truncated responses).
• Added a regression test (test_interrupted_multi_slice_checkpoint_is_partial_and_missed_on_reload); verified it fails with AssertionError on the pre-fix llm.py and passes with the fix.
• Kept the PR description synchronized with the final implementation.
• Documented any limitations / unsupported cases explicitly (see below).

How was this tested?

Reproduced #4943's report first on the plain v8 checkout with the issue's mock harness (system Python 3.13-compatible interpreter, extract_files_direct patched; one slice succeeds, the second raises KeyboardInterrupt):

before fix: {"attempted_chunks": 2, "cache_entries": 1, "partial_flag": false, "warm_uncached_files": 0, "tail_present": false}
after fix:  {"attempted_chunks": 2, "cache_entries": 1, "partial_flag": true,  "warm_uncached_files": 1, "tail_present": false}

Then:

1. New regression test test_interrupted_multi_slice_checkpoint_is_partial_and_missed_on_reload (in tests/test_chunking.py): passes with the fix, AssertionError without it.
2. Existing checkpoint tests re-run after the change and still pass: test_checkpoint_scopes_cache_writes_to_chunk_files, test_truncated_chunk_is_cached_partial_and_missed_on_reload (driven individually, since the full pytest suite is not runnable in my environment — the only stubbed import was tree_sitter, which these paths do not execute; the mocked-endpoint paths above are unaffected by it).
3. Success-path guard (no regression for completed runs): a two-slice corpus where every chunk succeeds, followed by the CLI's final full save, leaves a clean non-partial entry that check_semantic_cache serves warm, including the tail-section nodes.

Limitations: the partial stamp makes a killed-after-chunks-but-before-final-save run re-extract the sliced files next run — conservative by design, same recovery cost a truncated-response slice already incurs. No cache-format change, so old entries remain readable.

Graphify-specific checklist

• I updated generated skill artifacts (uv run python -m tools.skillgen --bless) when changing their source fragments. — N/A: graphify/llm.py's _checkpoint_chunk is runtime code, not a skillgen source fragment (verified the generated skill-*.md artifacts do not embed this region), so no artifact regeneration is needed.
• I confirmed that AST/structural extraction remains deterministic (no ambient state dependencies like ENV variables).
• I reviewed changes for security implications (no unsafe interpolation into shell/Python).
• I confirmed no API keys or local-only graph data are included.
• (If applicable) I disclosed AI authorship in my commit messages (Co-authored-by trailer, per the repo's authorship convention).

[github-actions]

Thanks for the pull request, @rahul05ranjan. A maintainer will review it soon.

Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions.

A couple of things that speed up review: make sure the test suite passes on Python 3.10 and 3.13, and that the change keeps extraction deterministic.

[rahul05ranjan]

Closing in favor of #4947 — it landed first and covers strictly more: I reproduced the interrupted-slice case with the issue's mock harness (partial_flag false → true, warm_uncached 0 → 1 on v8), and #4947's approach (hold a sliced parent's results until its last slice returns, stamp failed-slice parents partial) fixes the interrupted path plus the failed-slice path my interim-partial stamping didn't address. Happy to help review if useful.

[graphify-labs]

Graphify reviewed this change.

No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.

Not checked: tests were not run; formal verification proved 0 of 1 changed function(s) (1 not verified).

Formal verification. PR-changed functions: 0/1 verified (0 proven, 0 may-equivalent, 0 distinguished) · 1 not verified (1 unsupported).

Not verified on this run: extract\_corpus\_parallel (unsupported).

---

Graphify review — findings

Stops extract_corpus_parallel from checkpointing a partly extracted oversized file as complete. When a file is split into several FileSlice chunks, each chunk's checkpoint now stamps the parent file as partial unless all of its slices are in that chunk. If the run dies mid-way (crash, kill, Ctrl-C), the next run misses the cache and re-dispatches the file instead of serving truncated content. A finished run still overwrites the stamp with a clean entry in its final save, and a new test interrupts after the first slice and checks that the cache reload misses.

No blocking issues surfaced. 2 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1181 functions depend on the 303 functions this change touches.

Health — this change adds coupling hotspots:

• worse: extract_corpus_parallel() — 29 callers, 12 callees

Verification — 1181 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 801 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

59 of 489 test file(s) selected (12%) via static blast radius.

• tests/test_backend_env_isolation.py — impact
• tests/test_backend_extras.py — impact
• tests/test_backend_opt_in.py — impact
• tests/test_build.py — impact
• tests/test_build_located_semantic_identity.py — impact
• tests/test_build_merge_dedup_scope.py — impact
• tests/test_build_merge_hyperedges_and_prune.py — impact
• tests/test_build_merge_shrink_guard.py — impact
• tests/test_c_include_resolution.py — impact
• tests/test_carried_hyperedge_remap.py — impact
• tests/test_charmap_encoding.py — impact
• tests/test_chunking.py — impact, changed-test
• tests/test_claude_cli_backend.py — impact
• tests/test_corrupt_graph_json.py — impact
• tests/test_cross_extension_reexport_self_cycle.py — impact
• tests/test_dedup.py — impact
• tests/test_dedup_remaps_hyperedges.py — impact
• tests/test_dedup_shrink_refuses_force_write.py — impact
• tests/test_dedup_survivor_richness.py — impact
• tests/test_evidence_binding.py — impact
• tests/test_ext_tagged_id_clash.py — impact
• tests/test_file_id_collision_4854.py — impact
• tests/test_file_slice.py — impact
• tests/test_file_truncation_warning.py — impact
• tests/test_global_graph.py — impact
• tests/test_go_qualified_resolution.py — impact
• tests/test_hyperedge_member_shapes.py — impact
• tests/test_image_vision.py — impact
• tests/test_injection_sentinel_coverage.py — impact
• tests/test_issue_3472_source_file_collision.py — impact
• tests/test_js_member_collisions.py — impact
• tests/test_label_retry.py — impact
• tests/test_labeling.py — impact
• tests/test_llm_backends.py — impact
• tests/test_llm_null_token_counts.py — impact
• tests/test_llm_parser.py — impact
• tests/test_llm_parser_reasoning.py — impact
• tests/test_merge_sweeps_reextract_orphans.py — impact
• tests/test_no_dedup_flag.py — impact
• tests/test_non_string_node_ids.py — impact
• tests/test_ollama.py — impact
• tests/test_ollama_ctx_sizing.py — impact
• tests/test_ollama_live.py — impact
• tests/test_ollama_native_chat.py — impact
• tests/test_ollama_retry_cap.py — impact
• tests/test_oversized_document_slicing.py — impact
• tests/test_partial_cache.py — impact
• tests/test_pdf_slicing.py — impact
• tests/test_pdf_token_estimate.py — impact
• tests/test_provider_registry.py — impact
• … and 9 more

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Formal verification

Could not verify: Could not verify extract\_corpus\_parallel.

The verifier did not have enough to check extract\_corpus\_parallel, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: no capturable inputs from the test suite; property tier: parameter `on_chunk_done` is annotated `Callable | None` — outside the synthesizable primitive/collection set

· 1 more finding(s) on lines outside this diff (see the check run).