Skip to content

feat(openai): wire per-call response_format and tool strict knobs thr…#857

Open
Sizolity wants to merge 4 commits into
cloudwego:mainfrom
Sizolity:feat/openai-structured-output-knobs
Open

feat(openai): wire per-call response_format and tool strict knobs thr…#857
Sizolity wants to merge 4 commits into
cloudwego:mainfrom
Sizolity:feat/openai-structured-output-knobs

Conversation

@Sizolity

@Sizolity Sizolity commented May 28, 2026

Copy link
Copy Markdown

What

Add two narrowly-scoped, fully-typed knobs to libs/acl/openai so callers can use OpenAI / OpenAI-compatible Structured Outputs features per-call, without dropping to untyped WithExtraFields workarounds or WithRequestPayloadModifier post-hoc JSON patching.

  1. WithResponseFormat(*ChatCompletionResponseFormat) model.Option — per-call counterpart of Config.ResponseFormat. Per-call value takes precedence over config; not supplying it preserves existing fallback behavior.
  2. ToolInfoExtraKeyStrict constant — opt a single tool into Structured Outputs strict mode via schema.ToolInfo.Extra[openai.ToolInfoExtraKeyStrict] = true. Non-bool values are silently ignored.
    Both implement what the existing libs/acl/openai/README.md already documents (Available call options → openai.WithResponseFormat(format); "JSON Schema Response" example referencing strict tools) but which currently has no working code path.

Why

Closes #856.
A common pattern is a single shared ChatModel that serves two kinds of calls:

  • A structured-extraction call that needs response_format: {\"type\":\"json_object\"} (or json_schema) so the server guarantees a syntactically valid JSON body.
  • A free-form text call on the same model that must NOT have response_format set.
    For tool calls, opt-in strict: true solves a parallel issue: without it, providers occasionally emit syntactically invalid JSON in tool-call arguments — including unescaped control bytes inside string values — which strict JSON parsers reject with errors like Syntax error at index N: invalid char.
    Today neither knob has a per-call switch via the typed API surface. See [openai] Structured-output knobs not wired through: WithResponseFormat (documented but missing) and FunctionDefinition.Strict (dropped during marshal) #856 for the full evidence trail.

Scope

This PR only touches libs/acl/openai:

File Change
option.go Add ResponseFormat field to openaiOptions; add WithResponseFormat setter; add ToolInfoExtraKeyStrict constant.
chat_model.go In genRequest, prefer specOptions.ResponseFormat over c.config.ResponseFormat. In toTools, read ti.Extra[ToolInfoExtraKeyStrict] as bool. In tool marshal site, propagate t.Function.Strict to openai.FunctionDefinition.Strict.
tool.go Add Strict bool to internal functionDefinition.
chat_model_test.go 6 new subtests under Test_genRequest.
option_test.go Setter test matching the existing pattern.
Design choice for the Strict knob: sourced via schema.ToolInfo.Extra (option 3 in #856's "Proposed direction (b)"). This keeps the change inside libs/acl/openai and does NOT require modifying cloudwego/eino core — schema.ToolInfo's public API is unchanged. If the team later prefers a typed field on schema.ToolInfo.Function, migrating is a one-line change to toTools.
Following the repo's convention (see #847), the matching re-export in components/model/openai/option.go for WithResponseFormat is left to a follow-up bump PR once a new libs/acl/openai is tagged. The ToolInfoExtraKeyStrict constant is already accessible to wrapper users by importing libs/acl/openai directly.

Tests

```
GOTOOLCHAIN=go1.24.4 MOCKEY_CHECK_GCFLAGS=false \
go test -count=1 -gcflags='all=-N -l' ./libs/acl/openai/...
ok github.com/cloudwego/eino-ext/libs/acl/openai
```
New subtests under Test_genRequest:

  • WithResponseFormat sets format when config has none
  • WithResponseFormat overrides Config.ResponseFormat (including JSONSchema + Strict round-trip)
  • no WithResponseFormat falls back to Config.ResponseFormat (regression pin)
  • ToolInfo.Extra openai_strict=true propagates to FunctionDefinition.Strict
  • tool without Extra leaves Strict=false
  • non-bool openai_strict in Extra is silently ignored
    Plus Test_OpenAIOptions_Setters/WithResponseFormat.

Backwards compatibility

All existing tests pass unchanged. Tools without Extra have Strict=false (pre-PR behavior). Calls without WithResponseFormat fall through to the prior Config.ResponseFormat path.
Closes #856.

…ough

Add two narrowly-scoped, fully-typed knobs to libs/acl/openai so
callers can use OpenAI / OpenAI-compatible Structured Outputs features
per-call without dropping to untyped WithExtraFields workarounds or
WithRequestPayloadModifier post-hoc JSON patching:
1. WithResponseFormat(*ChatCompletionResponseFormat) model.Option —
   per-call counterpart of Config.ResponseFormat. Per-call value takes
   precedence over config; not supplying it preserves existing fallback
   behavior (pinned by a regression test).
2. ToolInfoExtraKeyStrict — public constant for opting a single tool
   into Structured Outputs strict mode via
   schema.ToolInfo.Extra[openai.ToolInfoExtraKeyStrict] = true.
   Non-bool values are silently ignored. Sourced via Extra so no
   changes to schema.ToolInfo in cloudwego/eino core are required.
Both implement what libs/acl/openai/README.md already documents but
which currently has no working code path.
Closes cloudwego#856.
@CLAassistant

CLAassistant commented May 28, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

[openai] Structured-output knobs not wired through: WithResponseFormat (documented but missing) and FunctionDefinition.Strict (dropped during marshal)

3 participants