Skip to content

Claude Code compatibility: inline system role, stream keepalive, Kiro refusals - #1

Draft
italocjs wants to merge 3 commits into
mainfrom
fix/claude-code-compat
Draft

italocjs wants to merge 3 commits into
mainfrom
fix/claude-code-compat

Conversation

@italocjs

Copy link
Copy Markdown
Owner

Everything this gateway needed to run Claude Code (2.1.304) reliably against Kiro, collected on one branch. Based on upstream main @ a5292ca.

Commits

1. fix(anthropic): accept messages[].role system and merge into system prompt (upstream jwadow#197, by @W0n9)

Cherry-picked unchanged, author kept. Newer Claude Code sends inline role: "system" messages; without this the gateway answers 422.

2. fix(streaming): send keepalive while the client stream is silent (upstream jwadow#288 / issue jwadow#287)

Symptom: a large tool call (e.g. Write of a big file) stalls with "Waiting for API response … check your network", then Claude Code retries the whole turn.

Why: tool calls are buffered until Kiro finishes the response, so the client gets zero bytes for minutes. Claude Code's byte-level idle watchdog (CLAUDE_BYTE_STREAM_IDLE_TIMEOUT_MS, default 180-300 s) then aborts.

Fix: a ping event (Anthropic) or : keepalive SSE comment (OpenAI) every STREAM_KEEPALIVE_INTERVAL seconds (default 10, 0 disables) while the stream is silent.

3. fix(streaming): surface Kiro refusals instead of empty responses (new)

Symptom: replies come back empty. Claude Code shows "No response requested." or sends "Your previous response had no visible output", and sessions feel stuck or very slow.

Why: Kiro now refuses the gateway's fake-reasoning prompt in any conversation with history:

{"stopDetails":{"refusal":{"category":"REASONING_EXTRACTION","explanation":"The selected model cannot continue this conversation. ..."}},"stopReason":"CONTENT_FILTERED"}

The parser had no pattern for stopDetails, so this became an empty end_turn. Reproduced on both claude-sonnet-5.5 and claude-opus-5.5. A one-message request works; user → assistant → user is refused every time; the same with thinking: disabled works.

Fix: parse the refusal and report it the way each API reports a refusal:

  • Anthropic: stop_reason: "refusal", with the explanation as a text block.
  • OpenAI: finish_reason: "content_filter", with the explanation as content.
  • Both streaming and non-streaming.
  • When the category is REASONING_EXTRACTION and fake reasoning is on, the message and a WARNING log line say to set FAKE_REASONING=false.

Deliberately not an error event: Claude Code retries api_error, which would turn this into a retry loop.

Required configuration

FAKE_REASONING=false. With it, multi-turn requests answer normally; the refusal fix only makes the failure visible.

Testing

  • Full unit suite: 1726 passed. New tests: 15 for keepalive, 17 for refusals (parser, core, Anthropic and OpenAI, streaming and non-streaming). 7 of the refusal tests fail on the unpatched code; the other 3 check that normal behavior is unchanged.
  • Keepalive, live: pings every 10 s during a 40-50 s buffered tool call. Claude Code with CLAUDE_BYTE_STREAM_IDLE_TIMEOUT_MS=15000 finished a ~54 s Write (662 lines) without a retry.
  • Refusal, live, against the real Kiro API with FAKE_REASONING=true: the client now receives stop_reason=refusal with Kiro's explanation and the fix hint, instead of an empty reply.

Note on prompt caching

Kiro applies prompt caching automatically; no gateway change is needed. Measured from Kiro's meteringEvent on a growing conversation with a ~45k-token prefix:

Model Turn 1 Turn 2 Turn 3
Sonnet 0.616 credits 0.186 0.201
Opus 0.900 credits 0.351 0.293

Kiro's cachePoint field had no effect. The cache_read_input_tokens: 0 that clients see is only the gateway's own token estimate.

W0n9 and others added 3 commits October 19, 2026 23:31
…rompt

Claude Code sends SessionStart hook context as inline system messages per
the updated Anthropic Messages API. Gateway previously rejected these with
422 before conversion. Extract system-role messages and append them to the
top-level system field instead of treating them as user turns.

Co-authored-by: Cursor <cursoragent@cursor.com>
Tool calls are buffered until Kiro finishes the whole response, so a large
tool input (e.g. Claude Code writing a big file) leaves the client without a
single byte for minutes. Clients with an idle-stream watchdog treat that as a
dead connection and abort: Claude Code shows "Waiting for API response ...
check your network" and retries the whole turn.

Add with_keepalive() in streaming_core and wrap both streaming paths:
- Anthropic: `event: ping` (the API's own keepalive, skipped by SDKs)
- OpenAI: an SSE comment line (`: keepalive`), ignored per the SSE spec

Keepalives start only after the first real chunk, so errors raised before
the stream starts are unchanged. A client disconnect cancels the pending
read and closes the upstream stream. New STREAM_KEEPALIVE_INTERVAL setting
(default 10s, 0 disables). Non-streaming responses are not affected.
Kiro can block a response and send only a refusal event, e.g.
{"stopDetails":{"refusal":{"category":"REASONING_EXTRACTION",...}},
"stopReason":"CONTENT_FILTERED"}. The parser had no pattern for it, so the
client got an empty end_turn. Claude Code then shows "No response requested."
or nudges the model for "no visible output", which loops and looks like a
hang. REASONING_EXTRACTION is triggered by the gateway's own fake reasoning
prompt in any conversation with history.

- parsers: parse stopDetails.refusal into a refusal event
- streaming_core: KiroEvent/StreamResult carry the refusal; the event's
  content is a user-facing message (with a FAKE_REASONING=false hint when
  the gateway caused it); logged as a warning
- Anthropic: stop_reason "refusal" with the message as a text block
- OpenAI: finish_reason "content_filter" with the message as content
- both streaming and non-streaming
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants