Skip to content

streaming: let a new handle snapshot a stream that was recreated - #93

Merged
raphael merged 1 commit into
mainfrom
fix-snapshot-recreated-stream
Nov 2, 2026
Merged

raphael merged 1 commit into
mainfrom
fix-snapshot-recreated-stream

Conversation

@raphael

@raphael raphael commented Nov 2, 2026

Copy link
Copy Markdown
Member

Problem

streaming.Stream.Snapshot returned ErrStreamDestroyed for any stream that had been destroyed and then created again, when the caller read through a new handle. Writes to that stream kept succeeding, so a caller saw a stream whose sequence numbers advanced while every snapshot came back empty.

Where it showed up: Chorus deletes a channel's stream when the channel's last member expires. A channel used again afterwards lives in stream generation 2. Chorus reads each channel through a new handle, and treats ErrStreamDestroyed as "nothing retained". So once the channel was recreated, read, watch and the dashboard all showed zero messages.

Cause

A recreated stream stores its events under a generation-scoped key, such as <key>:generation:2. A new handle is unbound: it has no generation yet and still carries the generation-1 key until its first operation binds it.

Snapshot's Lua script compared the handle's key with the stream's current key even when the handle was unbound. The publish (AddOnce) and replay scripts already check the generation and key only for a bound handle; the snapshot script was the one exception.

Change

The snapshot script now uses the same condition as replay: it checks the generation and the physical key only when the handle is bound. A handle still bound to a destroyed generation is still rejected.

TestSnapshotReadsRecreatedStreamFromNewHandle covers the case:

  1. destroy a stream and create it again (generation 2);
  2. read it from a new handle, which returns the new event and binds to generation 2;
  3. read it from a handle still bound to generation 1, which returns ErrStreamDestroyed.

Without the fix, step 2 fails with the exact error seen in production.

The full race suite and go vet pass. The change has no wire, key or format change, so mixed versions are safe.

A stream that is destroyed and later used again lives in a new
generation, stored under a generation-scoped key such as
"<key>:generation:2". A new, unbound Stream handle still carries the
generation-1 key until its first operation binds it. Snapshot's script
compared that key with the stream's current key even for an unbound
handle, so every Snapshot of a recreated stream from a new handle
returned ErrStreamDestroyed, and callers that read through fresh handles
saw an empty stream while writes kept succeeding.

The snapshot script now checks the generation and key only for a bound
handle, exactly as the replay and publish scripts already do. A handle
still bound to the destroyed generation is still rejected. The new test
destroys a stream, creates it again, and reads it from a new handle.
@raphael
raphael merged commit c857ed9 into main Nov 2, 2026
5 checks passed
@raphael
raphael deleted the fix-snapshot-recreated-stream branch November 2, 2026 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant