Skip to content

fix: serialize graph persistence and preserve dirty state for mutations occurring during a save - #13

Merged
haolpku merged 2 commits into
OpenDCAI:mainfrom
DivyamTalwar:codex/dm-03-graph-persist-races
Sep 17, 2026
Merged

haolpku merged 2 commits into
OpenDCAI:mainfrom
DivyamTalwar:codex/dm-03-graph-persist-races

Conversation

@DivyamTalwar

Copy link
Copy Markdown
Contributor

@haolpku

Summary

NetworkX graph persistence now captures immutable revisions, serializes writes per store, preserves dirty state when a mutation happens during a save, and uses a unique temporary file for each atomic replacement.

Root cause

The old worker iterated the live graph and unconditionally set _dirty = False after the write. A mutation could land after the worker captured the old graph but before that reset, leaving newer in-memory data absent from disk while the next persist returned early. Concurrent writes also shared one PID-derived temporary filename.

Change

  • Protect graph capture and mutation bookkeeping with an in-process state lock.
  • Serialize persist calls with an async lock.
  • Track a mutation revision and clear dirty state only when the saved revision is still current.
  • Write detached documents through unique same-directory temporary files and clean them up on failure.
  • Keep profile/source/lineage reconciliation removals under the same state lock.

Regression coverage

Barrier/event-driven tests cover mutation during save, overlapping saves, unique latest-file results, write failures, retry behavior, and existing graph traversal/reload behavior.

Verification: python -m pytest -q — 197 passing, 5 optional SDK tests skipped.

The patch is intentionally in-process. It does not claim cross-process locking, fsync durability, or a storage-format migration.

Please review the concurrency model and merge if CI is green.

@DivyamTalwar

Copy link
Copy Markdown
Contributor Author

@haolpku — Could you please review the deterministic graph persistence-race fix and merge it if CI confirms the full suite? The PR explicitly does not claim cross-process locking or fsync durability.

@haolpku
haolpku merged commit 863e974 into OpenDCAI:main Sep 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants