Skip to content

fix(session): prevent concurrent save failures - #447

Merged
steipete merged 2 commits into
openclaw:mainfrom
henkterharmsel:agent/atomic-session-write-paths
Jul 27, 2026
Merged

fix(session): prevent concurrent save failures#447
steipete merged 2 commits into
openclaw:mainfrom
henkterharmsel:agent/atomic-session-write-paths

Conversation

@henkterharmsel

@henkterharmsel henkterharmsel commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes an issue where concurrent saves of the same ACPX session could choose the same temporary pathname. One writer could rename the shared temporary file while another writer still expected it to exist, causing an ENOENT failure or allowing one save to overwrite another writer's temporary payload.

Why This Change Was Made

Atomic-write temporary paths now include a per-write UUID in addition to the target path. All session-store callers use the same helper, so concurrent writes retain atomic rename semantics without sharing temporary files.

User Impact

Concurrent runtime and checkpoint updates can persist the same session reliably without intermittent temporary-file collisions.

Evidence

  • Added a deterministic concurrent-save regression test that holds both writes before rename and verifies distinct temporary paths and a valid final record.
  • pnpm run check passes on Node 24.18.0 and pnpm 10.33.2.
  • 849 regular tests and 110 coverage tests pass; formatting, typecheck, lint, build, viewer checks, and coverage thresholds are green.
  • codex review and the project autoreview both reported no findings.

Real behavior proof

Behavior or issue addressed: Concurrent saves through the real file session store must not share temporary paths or fail with ENOENT.

Real environment tested: Linux checkout of this branch using Node 24.18.0 and pnpm 10.33.2. The probe imported the production createFileSessionStore implementation directly; it did not use Vitest, mocks, or snapshots.

Exact steps or command run after this patch: Created a temporary ACPX state directory, froze Date.now() to force every write into the same millisecond, and ran 50 rounds of eight concurrent store.save(record) calls. It then loaded the session through the public store, parsed the destination JSON directly, and checked for leftover .tmp files.

Evidence after fix:

concurrent_saves=400
loaded_session=sid-proof
json_schema=acpx.session.v1
temp_files_remaining=0
result=PASS

Observed result after fix: All 400 concurrent writes completed without an exception. The final session was readable through the public API, its JSON was valid, and no temporary files remained.

What was not tested: I did not interfere with persistence timing on a production gateway, because doing so would risk live session state. Windows and macOS were not tested locally.

Testing boundary

The race was reproduced deterministically through the real file-session-store implementation with controlled filesystem timing. I did not attempt to force the timing window against a production gateway because that would require interfering with live session persistence.

AI-assisted disclosure

This PR is AI-assisted. I reviewed the implementation and understand that uniqueness is added only to the temporary pathname; the destination path and atomic rename behavior are unchanged.

@henkterharmsel
henkterharmsel marked this pull request as ready for review July 13, 2026 20:44
@henkterharmsel
henkterharmsel requested a review from a team as a code owner July 13, 2026 20:44
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. labels Jul 13, 2026
@clawsweeper

clawsweeper Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 27, 2026, 10:06 AM ET / 14:06 UTC.

ClawSweeper review

What this changes

The branch introduces a shared UUID-suffixed temporary-path helper for atomic session and session-index writes, updates three persistence writers to use it, and adds concurrent-save regression coverage.

Merge readiness

⚠️ Ready for maintainer review - 1 item remains

This PR remains necessary: current main still uses timestamp-only temporary names in the affected session persistence writers, while the branch gives every atomic write its own UUID-suffixed temporary path. The focused implementation, regression coverage, and supplied real-runtime evidence support ordinary maintainer review rather than cleanup closure.

Priority: P2
Reviewed head: 2505217507837b3bf1eda6651eb28d56ac13122a

Review scores

Measure Result What it means
Overall readiness 🦞 diamond lobster (5/6) A focused persistence repair with direct runtime proof, targeted regression coverage, and no concrete correctness or security finding.
Proof confidence 🦞 diamond lobster (5/6) Sufficient (live_output): The PR body includes a direct production file-store run showing 400 concurrent saves completed, the final session loaded successfully, JSON remained valid, and no temporary files were left behind.
Patch quality 🦞 diamond lobster (5/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (live_output): The PR body includes a direct production file-store run showing 400 concurrent saves completed, the final session loaded successfully, JSON remained valid, and no temporary files were left behind.
Evidence reviewed 5 items Current-main gap and proposed implementation: The supplied PR diff against current main replaces the timestamp-only temporary filename construction with createAtomicWriteTempPath, whose UUID suffix makes concurrent writes to the same destination use different temporary files.
All identified persistence writers use the shared helper: The branch updates the public file session store, internal session-record writer, and session-index writer, avoiding a partial fix where one persistence path could still collide.
Regression coverage targets the collision condition: The added runtime test freezes Date.now() and runs eight concurrent saves to the same session, then verifies that the record remains loadable and no temporary files remain; the helper unit test separately proves distinct per-write paths in one millisecond.
Findings None None.
Security None None.

How this fits together

ACPX persists session records and a session index as JSON files through file-backed stores. Each save writes a temporary file and renames it into place; unique temporary names prevent simultaneous saves from interfering before the rename completes.

flowchart LR
  A[Session runtime or checkpoint] --> B[File-backed session store]
  B --> C[Serialize record or index]
  C --> D[Create unique temporary path]
  D --> E[Write temporary JSON file]
  E --> F[Atomic rename to destination]
  F --> G[Readable session state]
Loading

Before merge

  • Complete next step (P2) - The branch already contains the concrete repair, regression coverage, and sufficient real-behavior proof; the remaining action is ordinary maintainer review, not an automated repair.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Persistence coverage 3 write sites updated, 1 shared helper added The patch consistently covers session records, public file-store records, and the session index instead of repairing only one collision path.
Patch scope 7 files affected; 67 additions, 3 deletions The change is small and localized to persistence behavior, tests, and its user-facing changelog entry.

Technical review

Best possible solution:

Merge the narrow shared-helper fix after normal maintainer review and completion of the remaining required checks, preserving the existing destination paths and atomic-rename behavior.

Do we have a high-confidence way to reproduce the issue?

Yes—source inspection provides a high-confidence reproduction path: hold Date.now() constant and concurrently save the same session through the file store on current main, where same-process writes construct the same temporary pathname.

Is this the best way to solve the issue?

Yes—the UUID is confined to the private temporary pathname, preserves the destination path and atomic rename model, and centralizes the behavior so the three identified writers cannot drift apart.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 9866c7608764.

Labels

Label justifications:

  • P2: This is a bounded session-persistence reliability bug that can intermittently fail concurrent saves but is not shown to be a release-blocking outage.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes a direct production file-store run showing 400 concurrent saves completed, the final session loaded successfully, JSON remained valid, and no temporary files were left behind.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes a direct production file-store run showing 400 concurrent saves completed, the final session loaded successfully, JSON remained valid, and no temporary files were left behind.

Evidence

What I checked:

  • Current-main gap and proposed implementation: The supplied PR diff against current main replaces the timestamp-only temporary filename construction with createAtomicWriteTempPath, whose UUID suffix makes concurrent writes to the same destination use different temporary files. (src/session/persistence/atomic-write.ts:3, 250521750783)
  • All identified persistence writers use the shared helper: The branch updates the public file session store, internal session-record writer, and session-index writer, avoiding a partial fix where one persistence path could still collide. (src/runtime/public/file-session-store.ts:51, 250521750783)
  • Regression coverage targets the collision condition: The added runtime test freezes Date.now() and runs eight concurrent saves to the same session, then verifies that the record remains loadable and no temporary files remain; the helper unit test separately proves distinct per-write paths in one millisecond. (test/runtime.test.ts:237, 250521750783)
  • Real behavior proof supplied: The PR body reports a direct production-store probe with 400 concurrent saves under a fixed clock, a readable resulting session, valid JSON schema, and no remaining temporary files. The PR is also labeled proof: sufficient. (250521750783)
  • Recent ownership and review activity: The latest branch commit, test(session): make temp-path regression deterministic, was authored by steipete and the PR was assigned to steipete on July 27, 2026, indicating active human handling rather than an abandoned or superseded branch. (test/runtime.test.ts:237, 250521750783)

Likely related people:

  • steipete: Authored the latest deterministic regression-test commit on this branch and was assigned the PR in the review timeline. (role: recent session-area contributor and review owner; confidence: high; commits: 250521750783; files: test/runtime.test.ts, src/session/persistence/atomic-write.ts)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (4 earlier review cycles)
  • reviewed 2026-07-13T20:57:21.699Z sha d77504d :: needs real behavior proof before merge. :: none
  • reviewed 2026-07-13T21:14:27.692Z sha d77504d :: needs maintainer review before merge. :: none
  • reviewed 2026-07-13T21:34:03.546Z sha d77504d :: needs maintainer review before merge. :: none
  • reviewed 2026-07-19T00:06:41.346Z sha d77504d :: needs maintainer review before merge. :: none

@henkterharmsel

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jul 13, 2026
@steipete steipete self-assigned this Jul 27, 2026
@steipete
steipete force-pushed the agent/atomic-session-write-paths branch from d77504d to 2505217 Compare July 27, 2026 14:02
@steipete
steipete merged commit 1d17a61 into openclaw:main Jul 27, 2026
11 checks passed
@steipete

Copy link
Copy Markdown
Contributor

Landed the atomic-write collision fix with contributor credit preserved.

The failure came from using only PID plus millisecond time for temporary filenames: concurrent same-process writes could select the same path, and the first rename removed it from the later writer. The final patch centralizes UUID-suffixed same-directory temp paths across the public runtime store, session repository, and session index.

  • Exact head: 2505217507837b3bf1eda6651eb28d56ac13122a
  • Merge commit: 1d17a61f7eb3feda6c66370d1f990c6f4216db51
  • Regression proof: deterministic fixed-ID temp-path test plus concurrent production-store saves
  • Focused tests: 12 passed
  • Branch AutoReview: clean at 0.99
  • Hosted proof: approved fork run https://github.com/openclaw/acpx/actions/runs/30273168889 passed Test, Mutation, Slophammer, Docs, and the full exact-head matrix

Thanks @henkterharmsel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants