Skip to content

spec: OpenAI-Compatible Native Tool Return - #115

Open
Leoyzen wants to merge 3 commits into
develop/agenticfrom
feat/openai-compatible-native-tool-return
Open

spec: OpenAI-Compatible Native Tool Return#115
Leoyzen wants to merge 3 commits into
develop/agenticfrom
feat/openai-compatible-native-tool-return

Conversation

@Leoyzen

@Leoyzen Leoyzen commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Subclass OpenAIChatModel in agentpool to send native list tool return content instead of JSON strings for OpenAI-compatible models (GLM-5, vLLM, etc.) whose chat templates natively render list-type tool message content.

Problem

pydantic-ai's OpenAIChatModel always serializes list-type tool return values to JSON strings via tool_return_ta.dump_json(value).decode(). For models like GLM-5 whose chat templates branch on m.content is string vs list, this causes:

  • Unnecessary JSON escaping (["result1", "result2"]'["result1", "result2"]')
  • Loss of structural semantics — single <|tool_return|> block with escaped JSON instead of multiple native blocks
  • No escape hatch (pydantic-ai issue #3888 is still open)

Solution

Create OpenAICompatibleModel(OpenAIChatModel) in agentpool:

  1. Override _map_user_message() — when openai_chat_tool_return_as_list flag is True and tool return content is a non-empty list with no files, emit list[ChatCompletionContentPartTextParam] instead of JSON string
  2. Profile flag openai_chat_tool_return_as_list: bool (defaults to False for backward compat)
  3. openai_* kwarg passthrough — any openai_* prefixed key in YAML kw_args is auto-merged into the profile dict with type coercion (string→bool for known boolean keys)
  4. ImportModelConfig support with base_url, api_key, tool_return_as_list as constructor params

YAML Example

model_variants:
  glm-5:
    type: import
    model: agentpool.models.openai_compatible.OpenAICompatibleModel
    kw_args:
      model_name: "glm-5"
      base_url: "https://open.bigmodel.cn/api/paas/v4/"
      api_key: "${OPENAI_API_KEY}"
      tool_return_as_list: "true"
      openai_system_prompt_role: "developer"
      openai_supports_strict_tool_definition: "false"

Artifacts

  • openspec/changes/openai-compatible-native-tool-return/proposal.md — motivation, problem, proposal
  • openspec/changes/openai-compatible-native-tool-return/design.md — 7 decisions with pseudocode
  • openspec/changes/openai-compatible-native-tool-return/specs/openai-compatible-model/spec.md — 5 requirements, 11 scenarios
  • openspec/changes/openai-compatible-native-tool-return/tasks.md — 3 task groups, 18 subtasks

References

Closes #112

@Million-mo 请实现这个 spec 🙏

Subclass OpenAIChatModel in agentpool to send native list tool return
content instead of JSON strings for OpenAI-compatible models (GLM-5, vLLM).
Includes profile flag openai_chat_tool_return_as_list, openai_* kwarg
passthrough for arbitrary profile overrides, and comprehensive test plan.

Closes #112
@Leoyzen
Leoyzen requested a review from Million-mo July 6, 2026 07:41
@Leoyzen
Leoyzen removed the request for review from Million-mo July 6, 2026 07:42
@gemini-code-assist

Copy link
Copy Markdown

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

)

* feat: implement OpenAICompatibleModel with native list tool return

Subclass OpenAIChatModel to emit native list tool return content
instead of JSON strings for OpenAI-compatible models (GLM-5, vLLM).

- OpenAICompatibleModel(OpenAIChatModel) with _map_user_message override
- Profile flag openai_chat_tool_return_as_list (default False)
- openai_* kwarg passthrough with string→bool coercion
- ImportModelConfig support with base_url, api_key, tool_return_as_list
- 25 tests covering all spec scenarios

Implements: openspec/changes/openai-compatible-native-tool-return/

* fix: validate openai_* kwargs before OpenAIProvider construction

Move profile override validation before OpenAIProvider init so invalid
kwargs raise TypeError without requiring OPENAI_API_KEY credentials.

Fixes CI failure in test_import_model_config_non_openai_kwarg_raises.
…117)

* fix: resolve flaky test_nested_subagents_create_recursive_toolparts

Add 200ms sleep before publishing depth=2 spawn event to ensure the
parent consumer is fully started. Without this, the event can be missed
on slow CI runners, causing the test to hang until timeout.

* fix: increase sleep in test_child_done_events_timeout_continues

Increase wait from 150ms to 300ms to avoid timing flakiness on slow
CI runners where 50ms timeout + two turn executions exceed 150ms.

* refactor: use polling loop instead of hardcoded sleep for race condition

Replace fixed 200ms sleep with a polling loop that waits for the parent
consumer to register in integration._contexts before publishing the
depth=2 spawn event. Faster on local machines, more robust on slow CI.

Accepts Gemini Code Assist review suggestion.
@Leoyzen

Leoyzen commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

@Million-mo 这里我们需要在重构中考虑,去掉llmming-models的依赖,是直接使用pydantic-ai models 定义,还是我们自己封装一套?尤其是对于模型更细粒度的配置,包括标示是否支持多模态等

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pydantic-ai OpenAI tool return list content 被序列化为 JSON 字符串,与 OpenAI-compatible 模型(如 GLM-5)的 chat template 不兼容

2 participants