feat(openai): wire per-call response_format and tool strict knobs thr…#857
Open
Sizolity wants to merge 4 commits into
Open
feat(openai): wire per-call response_format and tool strict knobs thr…#857Sizolity wants to merge 4 commits into
Sizolity wants to merge 4 commits into
Conversation
…ough Add two narrowly-scoped, fully-typed knobs to libs/acl/openai so callers can use OpenAI / OpenAI-compatible Structured Outputs features per-call without dropping to untyped WithExtraFields workarounds or WithRequestPayloadModifier post-hoc JSON patching: 1. WithResponseFormat(*ChatCompletionResponseFormat) model.Option — per-call counterpart of Config.ResponseFormat. Per-call value takes precedence over config; not supplying it preserves existing fallback behavior (pinned by a regression test). 2. ToolInfoExtraKeyStrict — public constant for opting a single tool into Structured Outputs strict mode via schema.ToolInfo.Extra[openai.ToolInfoExtraKeyStrict] = true. Non-bool values are silently ignored. Sourced via Extra so no changes to schema.ToolInfo in cloudwego/eino core are required. Both implement what libs/acl/openai/README.md already documents but which currently has no working code path. Closes cloudwego#856.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add two narrowly-scoped, fully-typed knobs to
libs/acl/openaiso callers can use OpenAI / OpenAI-compatible Structured Outputs features per-call, without dropping to untypedWithExtraFieldsworkarounds orWithRequestPayloadModifierpost-hoc JSON patching.WithResponseFormat(*ChatCompletionResponseFormat) model.Option— per-call counterpart ofConfig.ResponseFormat. Per-call value takes precedence over config; not supplying it preserves existing fallback behavior.ToolInfoExtraKeyStrictconstant — opt a single tool into Structured Outputs strict mode viaschema.ToolInfo.Extra[openai.ToolInfoExtraKeyStrict] = true. Non-bool values are silently ignored.Both implement what the existing
libs/acl/openai/README.mdalready documents (Available call options →openai.WithResponseFormat(format); "JSON Schema Response" example referencing strict tools) but which currently has no working code path.Why
Closes #856.
A common pattern is a single shared
ChatModelthat serves two kinds of calls:response_format: {\"type\":\"json_object\"}(orjson_schema) so the server guarantees a syntactically valid JSON body.response_formatset.For tool calls, opt-in
strict: truesolves a parallel issue: without it, providers occasionally emit syntactically invalid JSON in tool-call arguments — including unescaped control bytes inside string values — which strict JSON parsers reject with errors likeSyntax error at index N: invalid char.Today neither knob has a per-call switch via the typed API surface. See [openai] Structured-output knobs not wired through: WithResponseFormat (documented but missing) and FunctionDefinition.Strict (dropped during marshal) #856 for the full evidence trail.
Scope
This PR only touches
libs/acl/openai:option.goResponseFormatfield toopenaiOptions; addWithResponseFormatsetter; addToolInfoExtraKeyStrictconstant.chat_model.gogenRequest, preferspecOptions.ResponseFormatoverc.config.ResponseFormat. IntoTools, readti.Extra[ToolInfoExtraKeyStrict]as bool. In tool marshal site, propagatet.Function.Stricttoopenai.FunctionDefinition.Strict.tool.goStrict boolto internalfunctionDefinition.chat_model_test.goTest_genRequest.option_test.goschema.ToolInfo.Extra(option 3 in #856's "Proposed direction (b)"). This keeps the change insidelibs/acl/openaiand does NOT require modifyingcloudwego/einocore —schema.ToolInfo's public API is unchanged. If the team later prefers a typed field onschema.ToolInfo.Function, migrating is a one-line change totoTools.components/model/openai/option.goforWithResponseFormatis left to a follow-up bump PR once a newlibs/acl/openaiis tagged. TheToolInfoExtraKeyStrictconstant is already accessible to wrapper users by importinglibs/acl/openaidirectly.Tests
```
GOTOOLCHAIN=go1.24.4 MOCKEY_CHECK_GCFLAGS=false \
go test -count=1 -gcflags='all=-N -l' ./libs/acl/openai/...
ok github.com/cloudwego/eino-ext/libs/acl/openai
```
New subtests under
Test_genRequest:WithResponseFormat sets format when config has noneWithResponseFormat overrides Config.ResponseFormat(including JSONSchema + Strict round-trip)no WithResponseFormat falls back to Config.ResponseFormat(regression pin)ToolInfo.Extra openai_strict=true propagates to FunctionDefinition.Stricttool without Extra leaves Strict=falsenon-bool openai_strict in Extra is silently ignoredPlus
Test_OpenAIOptions_Setters/WithResponseFormat.Backwards compatibility
All existing tests pass unchanged. Tools without
ExtrahaveStrict=false(pre-PR behavior). Calls withoutWithResponseFormatfall through to the priorConfig.ResponseFormatpath.Closes #856.