[guanbear]

Problem

On the runtime endpoint (runtime.<region>.kiro.dev) there is no /ListAvailableModels API, so /v1/models is served entirely from the static FALLBACK_MODELS list. Models not yet included in that list may work when requested directly — the account manager even "learns" them into its routing map on success — but they never appear in /v1/models, which breaks every client that relies on the endpoint for model discovery.

Approach

Register a previously unadvertised model in its selected account's cache only after a response body is collected successfully or a streaming response completes successfully. This applies to OpenAI and Anthropic APIs in both account-system and legacy modes. An initial HTTP 200 alone is insufficient: failed bodies, failed streams, rejected requests and client disconnects do not register the model.

• ModelInfoCache.add_discovered_model() creates idempotent discovered metadata with default token limits.
• Cache refresh preserves confirmed discoveries for the process lifetime, including after their metadata is replaced by the upstream catalog. Fresh upstream metadata takes precedence; unrelated stale advertised models are still removed.
• The existing account manager routing, sticky-account and success-statistics timing are unchanged. Registration happens at the successful response-completion points in the routes.

No probing, extra upstream requests, configuration or dependencies are introduced. Accounts keep separate caches, and /v1/models exposes confirmed discoveries through the existing aggregation/resolver path.

Verification

Current upstream base: a5292ca04c7c, checked 2026-10-08.

• 244 affected tests pass when the existing files run in isolated processes: cache/account manager 59, OpenAI routes 91, Anthropic routes 94.
• 31 new tests exercise the actual route functions with mocked upstream I/O and real account caches, including API × account mode × streaming mode success/failure paths, cancellation/disconnect, /v1/models visibility, refresh persistence, upstream metadata priority and account isolation.
• The full unit run passed 1625 tests and encountered 79 fixture setup errors (No accounts configured in credentials.json). The unchanged base passed 1594 tests and encountered the exact same 79 errors. The previously documented four truncation tests also have an existing cross-file order dependency; both API test files pass independently.
• Spec and Standards reviews report no remaining actionable findings; the final cache persistence edge case received an additional read-only review.
• PR whitespace/diff checks pass.
• No new live upstream inference was performed in this update. The following is the earlier dated end-to-end result.

End-to-end (2026-09-20, IAM Identity Center account via kiro-cli SQLite, runtime.us-east-1.kiro.dev, legacy mode):

step	result
GET /v1/models	13 models, no claude-opus-5
POST /v1/chat/completions with claude-opus-5	200 OK
GET /v1/models again	14 models incl. claude-opus-5; log: Discovered new model via successful request: claude-opus-5

Notes

• This makes the model list self-maintaining instead of requiring a release every time Kiro ships a model. It complements static additions: statically listed models appear immediately at startup, discovered models after first successful use.
• Alternative/sibling PRs fix(config): add claude-opus-4.8 to FALLBACK_MODELS #242, feat(models): add claude-opus-5 to fallback model list #279 and part of feat(models): add Claude Sonnet 5 support #260 add specific IDs statically; this mechanism makes that class of PR unnecessary going forward.
• Test suite: 4 tests in test_routes_anthropic.py (TruncationRecovery group) fail when run in the multi-file combination on main as well — pre-existing cross-file pollution, unrelated to this change (single-file runs pass; my branch shows the identical 4 failures + this PR's new test passing).

[cla-bot]

Thanks for the PR! 🎉

Before merge, we need a one-time CLA confirmation.
It confirms that you have the right to contribute this code and allow the project to use it.

Full CLA text:
https://github.com/jwadow/kiro-gateway/blob/main/CLA.md

Please reply once with:

I have read the CLA and I accept its terms

You need to write once, all further messages from me can be ignored.

[guanbear]

Thanks for the PR! 🎉

Before merge, we need a one-time CLA confirmation. It confirms that you have the right to contribute this code and allow the project to use it.

Full CLA text: https://github.com/jwadow/kiro-gateway/blob/main/CLA.md

Please reply once with:

I have read the CLA and I accept its terms

You need to write once, all further messages from me can be ignored.

I have read the CLA and I accept its terms

[guanbear]

I have read the CLA and I accept its terms

[ankitcharolia]

@guanbear could you give a try to this gateway: https://github.com/ankitcharolia/kiro-gateway

It works quite well with All AI harness and actively being developed. The most important thing is that it is ACP compliant

Models are automatically detected for claude, no need to manually update when new one is released

[cla-bot]

Thanks for the PR! 🎉

Before merge, we need a one-time CLA confirmation.
It confirms that you have the right to contribute this code and allow the project to use it.

Full CLA text:
https://github.com/jwadow/kiro-gateway/blob/main/CLA.md

Please reply once with:

I have read the CLA and I accept its terms

You need to write once, all further messages from me can be ignored.

[guanbear]

Thanks for maintaining the project. I reviewed this against the current main branch and fixed two edge cases: a model is now advertised only after its complete response succeeds, and confirmed discoveries survive subsequent cache refreshes while fresh upstream metadata takes priority. Existing routing and success-statistics behavior is preserved.

Both APIs and account modes are covered, including streaming/non-streaming, rejected requests, failed bodies, disconnects and account isolation. The 244 affected tests pass in separate processes. The full-unit run has the same 79 credential-fixture errors as unchanged main; details are recorded in the PR description. Spec and Standards reviews have no remaining actionable findings.

The CLA check still reports ERROR despite my existing acceptance comments on September 20 and 21. The log for this latest push is accessible and says that @guanbear has not signed the CLA; it appears the prior acceptance has not been recorded by the bot. Could you please check or re-run the CLA verification when convenient? I have not repeated the acceptance or changed the legal wording. Thanks for your help.

[guanbear]

Thanks for your help. I checked the current CLA error log and the main-branch .clabot configuration: the log reports that guanbear is not signed, and the configured contributors list does not contain my account. My acceptance comments from September 20 and 21 are already on this PR. Could you please record that existing acceptance and rerun the check? The bot asks for a one-time confirmation, so I have not posted a new acceptance. The code head remains c3a15b4; this follow-up is about the CLA gate only.

[guanbear]

A brief follow-up on the remaining CLA gate: the current head still reports verification/cla-signed: ERROR, and my existing acceptance comments from September 20 and 21 remain on this PR. Could you please record that acceptance in the contributor list or rerun the verification when convenient? No additional code changes have been pushed for this gate. Thank you for your help.