fix: preserve existing memory_units when update_mode=append#2785
Closed
handnewb wants to merge 2 commits into
Closed
fix: preserve existing memory_units when update_mode=append#2785handnewb wants to merge 2 commits into
handnewb wants to merge 2 commits into
Conversation
When update_mode='append' and the document already exists, the content_hash is always different (old+new text), so is_recovery stays False and handle_document_tracking cascade-deletes all existing chunks and memory_units on every retain. This change sets is_recovery=True when update_mode='append' and a document row exists, so upsert_document_metadata is used instead of handle_document_tracking. Old chunks whose content_hash hasn't changed are preserved alongside their memory_units. Closes vectorize-io#2664
Collaborator
|
see comment in the original issue, this is by design |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When
update_mode="append"is used with a stabledocument_id, the server-side implementation performs a full replace on every retain, silently deleting all existing memory_units for that document. This breaks the documented behavior and causes silent memory loss for integrations that use session-scoped document IDs (OpenClaw, Hermes, etc.).Root Cause
In the retain orchestrator,
is_recoveryis only set to True whencontent_hashmatches. Withupdate_mode="append", the content is always (old+new), so the hash never matches andhandle_document_tracking(cascade-delete old chunks + memory_units) is called instead ofupsert_document_metadata(preserve existing + add new).Fix
When
update_mode="append"and the document already exists, setis_recovery = Truesoupsert_document_metadatais used, preserving old chunks whose content_hash hasn't changed and their memory_units.9 lines added, zero existing behavior changed for non-append paths.
Impact
Closes #2664