Repository navigation
[Bug]: coding-agents — a fork re-retains the inherited transcript under a new refId, and can re-append a byte-identical injection block #5940
Description
[Bug]: coding-agents — a fork re-retains the inherited transcript under a new refId, and can re-append a byte-identical injection block
hindsight-integrations/coding-agents (0.8.2) keeps two pieces of per-session state only in memory, keyed by session id. A fork changes the session id and a restart empties the process, so both reset while the history they relate to is still there:
| State | Where it lives | What it tracks |
|---|---|---|
| retain cursor | core/retain-cursor.ts — retain-cursor.ts:194 |
how much of the transcript has been retained, plus server-side ack state |
| injection dedupe | const lastInjected = new Map<string, string>() — src/dsh.ts:249 |
"the last block text I appended for this session" |
Two symptoms follow. The first is server-side and happens on every fork; the second is model-visible and needs the recall text to be unchanged.
Symptom 1 — the inherited transcript is re-retained in full
The retain document identity is derived from the session id:
src/core/chat.ts:326 — const refId = `conversation:${sessionId}`;
and the retain operation id is a content hash that includes it:
// Identity of ONE payload: a resubmission of the same bytes is collapsed server-side into the
// original operation instead of extracting (or appending) twice.
const opId = (mode: string, content: string) =>
uuidV5(`${client.bank}\n${refId}\n${mode}\n${content}`);So a fork gets a different refId, and the same-bytes collapse cannot apply across the two.
The cursor is the only thing that would keep the first write small, and it is in memory (retain-cursor.ts:194). With no cursor for the child, the plan is mode: "replace" (retain-cursor.ts:162), i.e. the whole inherited transcript is sent as the child's first write and extracted again.
The same file's own note argues that this must not happen for a session whose parent already stores the text: src/dsh.ts:196-199 skips retention for subagent sessions because "retaining it would file a fragment of a session that the parent already stores in full". A fork is the same situation with a different label.
Symptom 2 — a byte-identical injection block can be appended again
dsh keeps injected messages in the history it sends ("so re-appending an unchanged block (the per-turn roster) only stacks identical copies (#5026)"), and dedupes them per session through lastInjected, compared by exact text (src/dsh.ts:281 consults the map, not the history).
Appending a changed block is intended behaviour — the parent session does the same on every turn whose recall text differs. The defect is the unchanged case: when the block text is identical to the last one appended (src/dsh.ts:287 appends unconditionally once the map says nothing), the history ends up carrying the same text twice. What is observable is "the same text twice", not two blocks in any deeper sense.
- Fork:
session/forkproduces a new session id, so the map has no entry for the child while the inherited history already contains the parent's block (logged as a user/message, ahead of the parent's last reply). When the child's recall text is unchanged from it (the [coding-agents][dsh] same ~911-char context block injected on every turn, inflating token usage #5026 roster case), the child's first step appends a byte-identical copy. dsh records the fork lineage on the header (isSeeded,parentSession;dsh-sessionSessionHeader) and the inherited prefix length on the session record (inheritedEventCount), and a fork seed replays the parent's event log — so the inherited block really is there. (From the host's seed semantics; not directly observed in a model request here.) - Restart: the map is rebuilt empty, so the same append happens after every restart whenever the recall text is unchanged.
Each append mints a fresh id: randomUUID() (src/dsh.ts:233), so the duplicate is a distinct message rather than a re-reference.
What this report does and does not establish
- Verified (upstream sources at
mainplus the shipped 0.8.2 bundle): both code paths above, and that dsh hands the plugin a new session id for a fork (session/forkreturns a newSessionId). - Not run: the opencode adapter, which uses the same
refIdderivation but was not exercised here. - Not measured against a live bank: whether the second write-back ends up as a second document. The claim is what the code path implies — a document keyed by
refIdwith no dedupe key shared with the parent — not an observed bank state. If the server does collapse these, this reduces to wasted extraction; if it does not, it is a duplicate document. - No prefix-cache claim is made: on dsh the injection is a message appended at the tail, so the inherited prefix still matches the parent's cache. The cost here is re-retention and context growth, not prefix loss. The prefix-cache break belongs to a different injection channel — see [Bug]: coding-agents: once-per-session
<hindsight_memory>block makes the system prompt unstable — breaks the provider prefix cache on turn 2 and on every fork #5914 under Related.
Steps to Reproduce (dsh)
-
Install the plugin the supported way:
dsh plugin --profile <profile> add @vectorize-io/hindsight-coding-agents.package.jsondeclaresdsh.bundle.patch, so the package's owncordis.patch.ymlis appended to the profile's ordered bundle list (package.json→dsh.profile.bundles) and inserts{ id: hindsight, name: "@vectorize-io/hindsight-coding-agents/dsh" }on the host plane. Memory settings live outside dsh, in~/.hindsight/coding-agent.json. -
Fork path (Symptom 1, unconditional): fork a session (
session/fork,atSeqomitted → last completed turn) and let one turn complete. Inspect the retain request in the diag trail or via a recording proxy:- the child's first write has
ref_id: conversation:<child-session-id>,session_id: <child-session-id>(chat.ts:385-391), acontentcontaining the whole inherited transcript, and no append mode; - the parent's most recent write, once its cursor exists and the append probe passed, has
updateMode: "append"and only its new turns.
The only mode-observability the plugin emits today is
diag(harness, "retain_mode", { mode, appendSupported })(chat.ts:345), which fires only when the append probe ran — aretain_modeline withmode: "replace"on the child's first turn is the cheap signal. - the child's first write has
-
Restart path (Symptom 2, deterministic): take a session with a recalled block whose text is stable across turns (the per-turn roster case from [coding-agents][dsh] same ~911-char context block injected on every turn, inflating token usage #5026). Restart the host and send a prompt whose recall text is unchanged — the history now carries the same block text twice, visible to the model.
-
Fork path (Symptom 2): same session, fork instead of restart, prompt whose recall text is unchanged from the parent's last block — the child's request carries the inherited block plus a byte-identical copy.
Expected Behavior
- A fork should reuse the parent's retain identity, or seed the cursor from the parent, so the inherited prefix is not extracted a second time.
- Neither a fork nor a restart should append a byte-identical copy of a block the history already contains.
Actual Behavior
- Every fork pays a full write-back of the inherited transcript against a new
refId, and therefore a full extraction of history the bank already holds. - On a fork or restart with unchanged recall text, a duplicate block is appended and stays in every later request; each restart can add one more.
Impact
Forking is the ordinary "branch a conversation" gesture, and restarts are unavoidable in normal use.
- Re-retain (every fork): a full transcript write-back plus re-extraction, and — expected from the code path, not verified against a live bank — a second document holding history the parent already stored, which then competes with the parent in recall results. Because the cursor is in memory, the same replay also happens on every process restart: not duplicate-producing for a same session — the replace rewrites one document — though it still re-sends and re-extracts the whole transcript (the cost coding-agents 0.8.1: a failed version probe disables append for the life of the client #5735 records), and for a fork it lands on a different
refId. - Duplicate block (conditional): the duplicate's tokens are carried in every subsequent request of that session, permanently, and each restart can add one more copy — the stack grows without bound across restarts. A fork whose history carries a duplicated memory block is a behavioural difference from the "same state" the host promises, not only a cost one.
Suggested Direction
Two repairs under one framing: both states are in-memory maps keyed by session id that a fork or restart resets.
- Retain cursor — seed it or persist it: it cannot be derived from the local transcript alone, because it also holds server-side acknowledgement state (
dirty,pending,opsinretain-cursor.ts). Seeding from the inherited prefix length the host already reports (inheritedEventCount) covers the fork case; persisting perrefIdcovers the restart case. - Injection dedupe — derive it from the history: the last message whose
source.kind === "plugin:hindsight"answers exactly the question the map is trying to remember, so the unchanged-text case cannot re-append.
The fork lineage is already in the hook payload, so neither needs new plumbing: dsh exposes parentSession / isSeeded on the session header. preStep is the reliable point (src/dsh.ts:266 reads agent.session.header.id); sessionStart reads it at :254, but "a resumed session gets no agent/session-start in some compositions". The package's own cordis.patch.yml states the row mounts on the host plane, which is where "the pre-step/session-start/turn-stopping events and the layered tools registry all reach every session" — so those three are the whole lifecycle surface available here.
If seeding the cursor is not done, two fallbacks for the re-retain symptom:
- Share the parent's
refId: on a seeded session, write the parent'srefIdso the child appends from the fork point. Note the failure mode: two sessions would then hold cursors for one document, and any laterreplacefrom either side rebuilds that document from its own transcript alone (chat.ts:346-348: "A replace rebuilds it from THIS file alone"), dropping the other branch's turns — reachable, because forks diverge. - Skip the inherited prefix: keep a per-child
refId, but plan the first write-back as anappendstarting at the inherited prefix length, so only post-fork turns are retained. This keeps each branch as its own document and is the safer default.
Version
@vectorize-io/hindsight-coding-agents0.8.2, installed as a dsh profile bundle (dsh plugin --profile <profile> add …; the bundle list ispackage.json→dsh.profile.bundles, and the row mounts on the host plane from the package's owncordis.patch.yml)- harness
dsh0.2.1-rc.2; mechanism verified by reading the shipped bundle and the upstream sources atmain(chat.tsb335b7f,retain-cursor.tsd3f0001,dsh.ts87a9d03, all 2026-10-12)
Related
- [Bug]: coding-agents: once-per-session
<hindsight_memory>block makes the system prompt unstable — breaks the provider prefix cache on turn 2 and on every fork #5914 — the cost side of the same session-identity keying on the opencode2 harness, where the recalled block goes into the system prompt and therefore does break the provider prefix cache on turn 2 and on every fork. The dsh path reported here injects a message at the tail, so it makes no prefix-cache claim; the two reports share the root cause (state keyed by session id, reset by a fork) at different injection channels. - coding-agents 0.8.1: a failed version probe disables append for the life of the client #5735 — the other path into
mode: "replace"(a failed append probe pins the client), and its "by design" note about in-memory cursors re-sending the full transcript on every process start. - [coding-agents][dsh] same ~911-char context block injected on every turn, inflating token usage #5026 — the stacked-copies note this report builds on for the unchanged-text case.
- Reflect's per-iteration tools narrowing (and final system-prompt swap) forces a full prefill on hybrid/Ollama models #3865 — a full prefill caused by per-iteration prompt assembly.
Activity
[-][Bug]: coding-agents — a forked session gets a new `refId`, so the inherited prefix is re-retained in full[/-][+][Bug]: coding-agents — a fork re-retains the inherited transcript under a new refId, and can re-append a byte-identical injection block[/+]