fix(claude): stabilize Responses instructions for Muse/Go prompt cache - #2
Closed
Warexpor wants to merge 7 commits into
Closed
fix(claude): stabilize Responses instructions for Muse/Go prompt cache#2Warexpor wants to merge 7 commits into
Warexpor wants to merge 7 commits into
Conversation
Strip growing <total_tokens> and TaskCreate footers from Claude Code system text before it becomes Responses instructions, and reattach the latest notice as a trailing input message so Muse/Go prefix cache can hit. Paper and measured rates live under docs/claude-instructions-cache-stabilize/. Co-authored-by: Warexpor <warexpor@gmail.com>
|
✅ Deterministic PR hygiene checks passed. |
The Claude Code path still uses metadata.user_id. Desktop has no session id, so the fallback key now fingerprints the cacheable prefix instead of raw systemParts that still contain <total_tokens> / TaskCreate footers. Co-authored-by: Warexpor <warexpor@gmail.com>
Restrict strip to standalone integer <total_tokens> footer lines and the exact Claude Code reminder paragraph so inline docs of those tags or TaskCreate stay in instructions. Co-authored-by: Warexpor <warexpor@gmail.com>
Identify Claude Code <total_tokens> / TaskCreate notices as a trailing suffix outside markdown fences. No match returns the original system text byte-for-byte so fenced examples and whitespace-only prompts stay untouched. Translator regressions cover those no-ops. Co-authored-by: Warexpor <warexpor@gmail.com>
Relocation runs only when metadata.user_id identifies Claude Code. Unmatched fence openers cover through EOF so an unfinished fenced example plus a numeric total_tokens line is not a harness suffix. Desktop keeps the raw systemParts cache-key fallback. Co-authored-by: Warexpor <warexpor@gmail.com>
Claude Code writes <total_tokens>N tokens left</total_tokens>, not a bare integer. Digit-only matching left the harness suffix in instructions and collapsed Muse/Go hit rate to baseline. Match that shape as a trailing unfenced suffix, run peel on every inbound translation (no-op when absent), and hash Desktop prompt_cache_key from the same string used for instructions. Co-authored-by: Warexpor <warexpor@gmail.com>
Keep the real N tokens left matcher and unclosed-fence-to-EOF handling. Relocation is now stabilizePromptCache (default false) on the translator, not metadata.user_id. Ordinary callers keep a matching suffix in instructions; /v1/messages passes true. Desktop prompt_cache_key hashes raw systemParts outside opt-in and the instructions string when opted in. Co-authored-by: Warexpor <warexpor@gmail.com>
|
@Ingwannu addressed on Kept the real
Verification: |
|
@Ingwannu addressed on Kept the real
Verification: inbound-cache-stabilize + inbound — 75 pass; |
Owner
Author
|
Duplicate of upstream lidge-jun#4052 — same head branch . Closing to avoid split review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
instructions. Growing<total_tokens>N tokens left</total_tokens>/ TaskCreate footers churn that prefix and collapse Muse/Go prompt-cache hit rates.stabilizePromptCache?: booleandefaults to false. Ordinary Anthropic callers with that exact unfenced suffix keepinstructionsandinputunchanged. The Claude Code/v1/messagesinbound path passes true. This is not inferred frommetadata.user_id.TRAILING_TOTAL_REfor<total_tokens>\d+\s+tokens left</total_tokens>, exact TaskCreate paragraph, trailing unfenced only, unclosed fence through EOF). No match returns the original string byte-for-byte.prompt_cache_keyfallback hashes rawsystemPartsoutside opt-in (previous key). When opted in, that fallback hashes the same string asbody.instructionsafter stabilize.SOCKS5 / Go session-header work is intentionally out of scope. Related Anthropic thinking-replay issue lidge-jun#3719 stays open (link only).
Upstream review PR: lidge-jun#4052
Verification
bun test tests/claude-integration/claude-inbound-cache-stabilize.test.ts tests/claude-integration/claude-inbound.test.ts— 75 passbun run typecheck— passChecklist