[Bug] Responses combo treats completed text as success despite tool_choice=required (NVIDIA target)

Client or integration

Direct HTTP/API client. Also affects Codex CLI when its RHODIZ combo selects the NVIDIA target.

Area

Proxy and routing; Streaming; Tools / MCP / web search.

Summary

A Responses request with a declared function and tool_choice: "required" can return HTTP 200, status: "completed" but zero function_call items when the selected target is NVIDIA Nemotron Ultra. The combo accepts that response as a successful target outcome, instead of identifying that the required-tool contract was unsatisfied. Consequently a Codex tool round-trip may make no progress, despite the client seeing a successful completion.

This differs from the existing issues about forwarding tool-choice settings on the Chat Completions path (e.g. #5211) and DeepSeek rejecting forced choices (#4631). This is a successful-but-semantically-invalid Responses result that a combo marks terminal-success.

Reproduction

Use a separately authorized OpenCodex instance with a configured NVIDIA provider; credential details are not needed in the report.

{
  "model": "nvidia/nvidia/nemotron-3-ultra-550b-a55b",
  "input": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_text",
          "text": "Call get_fixture_marker with key=status. Do not answer with text before the tool result."
        }
      ]
    }
  ],
  "tools": [
    {
      "type": "function",
      "name": "get_fixture_marker",
      "description": "Return a fixed test value",
      "parameters": {
        "type": "object",
        "properties": { "key": { "type": "string" } },
        "required": ["key"],
        "additionalProperties": false
      }
    }
  ],
  "tool_choice": "required",
  "store": false,
  "stream": false,
  "max_output_tokens": 750
}

Observed against the installed OpenCodex 2.89.0:

Route	HTTP	Response status	Output item types	Required function?
NVIDIA direct	200	completed	`reasoning`, `message`	No (0 calls)
GPT-6.1 Sol direct	200	completed	`function_call`	Yes (1 call)
Claude Opus 5.5 direct	200	completed	`function_call`	Yes (1 call)

With the exact same one-function request sent to the 11-target combo alias RHODIZ, the combo selected NVIDIA and returned completed without a function call. This reproduced 2/2 times. A separate 151-declared-function test reproduced the same outcome via the combo, whereas the GPT-6.1 Sol route completed the required function call and the subsequent function_call_output continuation 2/2 times.

An earlier 25-turn text-response stress test with 127/128/129/151 declared (but unused) tools had 25/25 HTTP 200 success. That test did not establish actual tool-call correctness: a real Codex CLI read-tool test subsequently showed intermittent failures on the combo (1/5 passes in one batch) versus GPT-6.1 Sol direct (5/5). This comparison is corroborating context, not proof of which exact model served every failed Codex CLI turn.

Version

@bitkyc08/opencodex 2.89.0, Codex CLI 0.163.0.

Operating system

Linux, x86_64 (VM; distribution not independently verified).

Provider and model

NVIDIA / nvidia/nemotron-3-ultra-550b-a55b; controls: openai/gpt-6.1-sol and anthropic/claude-opus-5-5.

Logs or error output

In src/server/responses/core-combo.ts, the combo returns a child immediately when response.ok after its stream preflight and success bookkeeping. The existing preflight in src/server/responses/combo-stream-preflight.ts detects certain retryable zero-output terminals, but does not enforce the caller's required-function selection contract for a completed 200 response. The upstream adapter may be disregarding required; the proxy currently does not detect that and makes the misbehavior a successful combo result.

See the same-issue streaming follow-up comment for per-event SSE counts. No raw user prompts or access logs are included.

Screenshots and supporting files

A minimal synthetic JSON reproducer and comparative results are embedded above. The follow-up comment documents streaming response.completed with zero calls and the isolated test counts. No sensitive attachments.

Redacted configuration

{
  "combos": {
    "rhodiz-coding-primary": {
      "alias": "RHODIZ",
      "strategy": "failover",
      "targets": [
        {"provider": "nvidia", "model": "nvidia/nemotron-3-ultra-550b-a55b"},
        {"provider": "openai", "model": "gpt-6.1-sol"},
        {"provider": "anthropic", "model": "claude-opus-5-5"}
      ]
    }
  }
}

The snippet is illustrative: the actual 11-target combo contains additional routes, whose details do not affect the minimal required-function reproduction.

Expected behavior / proposed acceptance

Preserve explicit tool_choice: "required" or a named mandatory tool at the upstream adapter boundary, and verify the upstream wire really supports it.
For a fully buffered, successful non-streaming Responses child, a completed response with zero required function_call items must not be reported as a valid completed tool round. Either safely select an eligible alternative before any output/effects, or return a typed compatibility error rather than a false success.
Streaming handling must remain conservative: never replay or hop after client-visible output, a tool side effect, or ambiguous commitment. Do not turn every text-only 200 into a retry. Handle only requests explicitly requiring a tool and only if it is provably safe.
Preserve complete tool catalogs (including 151 tools), tool call IDs, function_call_output continuation, caller model identity, authorization boundaries, and cancellation.
Add isolated positive/negative tests for direct and combo paths, no output vs. committed output, one required tool and 151 declared tools. A passing end-to-end sequence is: request with required → exactly one declared function call → caller-provided function result → completed answer using that result.

Privacy and provenance

Reported by the affected operator via their connected GitHub account. Examples use synthetic markers only; no account IDs, internal addresses, tokens, raw user prompts, or private logs are attached. The production installation has not been modified as part of this upstream report.

Checks

I searched existing issues and documentation.
I removed secrets, tokens, account details, request credentials, and personal data.

_{Automated translation bookkeeping — detected language: English.}

Issue reopened

The report now contains the information required by the automated check. Thanks for updating it.

Additional live evidence: SSE/streaming is affected too (OpenCodex 2.89.0)

A follow-up isolated read-only test sent the same one-function tool_choice: "required" request with stream: true. The function is a synthetic marker_func; nothing actually executes. All three routes returned HTTP 200 and response.completed:

Route	Terminal	Response item events	Function calls
NVIDIA direct	completed	2 `response.output_item.added`, 2 `.done`, `response.output_text.delta`	0
`RHODIZ` combo (selected NVIDIA)	completed	2 item-added, 2 item-done, `response.output_text.delta`	0
GPT-6.1 Sol direct	completed	1 item-added, 1 item-done, 5 function argument deltas and argument done	1 (`marker_func`)

Elapsed: NVIDIA direct ~14.1s; combo ~1.4s; Sol ~2.4s. No credential material or raw user content is included.

Important safety distinction: a local prototype guard that rejects fully buffered non-streaming missing-required-tool responses passed 20 isolated focused tests (52 assertions across compatibility/rollback-related test suites) and bundled successfully on 2.89.0. This prototype is staging only, not deployed and does not solve streaming. A general streaming fix must not replay a response once client-visible semantic output, a tool invocation, or ambiguous side effects may have been committed. Consider request-time eligibility/capability gating for providers that cannot honor required, and/or bounded pre-delivery inspection before publishing any SSE output, with exact positive/negative tests for the commitment boundary.

The actionable defect is not “every NVIDIA answer must include a tool”: it applies only when the caller explicitly requires a tool. Text-only tool_choice:"auto" responses must remain valid.

Additional implementation finding and tested mitigation (2026-10-21)

The streaming reproduction is not safely fixed by adding a post-dispatch SSE reader in combo-stream-preflight.ts alone. The actual NVIDIA openai-chat translation path produces a runTurnAdapterSseResponses-marked response, and core-combo.ts deliberately skips that preflight. The comment in core-lifetime.ts explains that the runTurn adapter owns an event queue and has already performed its own preflight; adding another reader could replay committed adapter work. A streaming implementation must respect this ownership boundary.

A narrow operator-side workaround was validated only in staging:

For the specific production combo rhodiz-coding-primary, when tool_choice == "required" and every declared tool is a client function (including a 151-function catalog), accept only the two destination models whose mandatory function-call round trips were independently verified: openai/gpt-6.1-sol and anthropic/claude-opus-5-5.
Do not change any configured combo target, provider credentials, tool catalog, tool_choice:"auto" request, hosted/unrecognized tool, compaction request, or other combo.
Perform filtering at request-time target selection, before any upstream request or output, not by replaying a text-only 200 after commitment.
Test simulation: if Sol is unavailable, Opus remains the next strict-function-capable target; other nine destinations remain available to ordinary requests.

Focused staging verification: 19 tests, 56 assertions passed, and Bun bundled the changed core-combo.ts successfully. It is intentionally a local compatibility gate rather than a proposed universal allowlist; upstream should provide provider/model capability reporting or an equivalent safe policy instead of hardcoding operator model IDs.

Since prioritizing Sol in the live 11-target combo using its authenticated administrative interface (with verified backup), the direct Responses acceptance suite completed 10/10 full tool cycles using 151 function declarations each: required function call → supplied function result → correct final answer. All twenty requests were served by Sol. The ordinary Codex CLI still has a separate intermittent pre-thread initialization/hang issue, so this is not an overall 10/10 system claim.

No source-code compatibility patch has been installed into the live OpenCodex service. The 11-target configuration is preserved.

Production acceptance update, 2026-10-21 — streaming control passes with verified Sol first

The operator applied the existing, officially authenticated combo-management change to place openai/gpt-6.1-sol first in the 11-target RHODIZ failover combo, preserving all nine original members in their relative order and anthropic/claude-opus-5-5 as the last target. Config persistence, backup, and post-write readback were verified.

Without deploying any source-code patch, real Responses API acceptance now passed:

10/10 non-streaming tool_choice:"required" cycles, each with 151 declared inert functions: correct function_call, client-supplied function_call_output, and correct final answer.
10/10 streaming SSE cycles of the same shape: response.output_item.done contains the required function call and response.output_text.delta reassembles to the exact test marker following the tool result. All 20 first/second streaming requests completed via gpt-6.1-sol.

This validates the operator's Sol-first mitigation while Sol is eligible. It does not resolve this issue's core bug: NVIDIA still independently returns 200/completed with no function call on required; if the combo can select that target after Sol is unavailable, the bad-success contract remains. The narrow request-time capability gate mentioned earlier remains staging-only; it was not installed.

For clarity, a previous streaming test script briefly produced false negatives because it inserted spaces while joining incremental response.output_text.delta chunks. We corrected the validator to concatenate deltas exactly, preserved the old faulty evidence separately, and the corrected ten-cycle run passed 10/10. This was a validator error, not an upstream model error.

Separate from this upstream issue, some standard-profile Codex CLI processes still hang before thread.started, even though the API tool round trips succeed and the official Codex daemon restart completes. No claim of full end-to-end system certification is made.

Reproduced against current dev (12025981e5a304cdb6b1b54f525ba8554f623977) with a synthetic 151-function catalog: the translated Chat adapter reports done for a text-only upstream completion even when the caller required a tool or selected a different named tool. The regression gives 16 passing controls and 4 expected failures across buffered JSON and SSE.

I am preparing a narrow fix at the existing adapter terminal boundary: retain output and usage, but end an unmet mandatory client-call result with typed required_tool_not_called failure. It does not replay visible output, hardcode a provider/model roster, or add a second preflight around runTurn responses. Route tests cover direct and combo dispatch and assert one physical request with no fallback after visible text. Native passthrough, runTurn capability policy and proxy sidecars are separate scopes; this is not a claim that the staging allowlist or a universal capability policy is ready to ship. I will link the Draft PR with final verification here.

Draft fix: #7463, commit e0587d08e2b874d1b19a6120b04f3ae18ed70552 against current dev 12025981e5a304cdb6b1b54f525ba8554f623977.

The new regression reproduced four failures before the patch. Final focused/adjacent checks passed 86 tests, plus 36 layout/ratchet checks; structure, ordinary privacy checks and the complete 585-page docs build passed. The route tests preserve the full synthetic 151-function call/result cycle and verify direct/combo JSON/SSE failures without replaying visible output.

Whole-project typecheck remains locally unverified after reaching the fixed five-minute resource cap; actual hosted CI and independent review are pending. The PR stays Draft. The fix covers the translated Chat client-call terminal only; proxy sidecars, native passthrough, runTurn routing and universal capability/alternate-target policy remain separate. This issue is not closed and the patch is not merged or released.

New live differential: `required` fails while explicit named selection succeeds (2.89.0)

RHODIZ operator follow-up, tested 2026-10-21 14:48 UTC against the unchanged production OpenCodex 2.89.0. This is live provider evidence, not validation of the unreleased #7463 patch. Only inert caller-supplied functions were declared; no filesystem or business actions were executed.

Route / selection	Catalog	Transport	Result
RHODIZ combo, `required` (actually served by Sol)	127, 128, 129, 151	buffered JSON	4/4 valid call → supplied output → correct marker
RHODIZ combo, `required` (actually served by Sol)	151	SSE	1/1 valid full cycle
NVIDIA Nemotron Ultra, `required`	1	JSON and SSE	0/2: HTTP 200, `status: completed`, zero `function_call`; SSE ends with `response.completed`
Same NVIDIA target, `{ "type": "function", "name": "get_fixture_marker" }`	1	JSON and SSE	2/2 valid full cycles, exact `{ "key": "status" }` arguments
Anthropic Opus direct, `required`	151	JSON and SSE	2/2 valid full cycles

New actionable distinction: on this live target, required and explicit named selection behaved differently with the same single-function catalog and prompt. This reinforces that the observed failure is not a >128-tool catalog limit. The two successful named controls are not evidence of universal NVIDIA tool compatibility, nor justification to silently choose a function for an arbitrary multi-tool required request.

The failing requests used model nvidia/nvidia/nemotron-3-ultra-550b-a55b, store: false, max_output_tokens: 1600, one function named get_fixture_marker with required string key, and the instruction to call it with key=status before answering. Both buffered and streaming validators inspect the actual terminal envelope and completed function items; no text is counted as a function call. Continuation validators use caller-supplied inert markers.

Total: 9 valid cycles / 11 cases, with 2 false-success protocol failures. Evidence JSONL SHA-256: 59bc0bd61dadeb1034ac792d617be6dee7b64fae7ab418208ed9cd4920fae3e9. Client configuration and client catalog hashes were unchanged; no runtime source patch, provider removal, or production combo change was made.

Please retain the single-function required versus named distinction as a regression/control when integrating #7463. A typed failure is the correct outcome after already-visible output; do not replay a committed stream. Any pre-dispatch capability/eligibility policy for reliable full-chain failover remains a separate acceptance gate. Our production 10/10 certification remains withheld.

fix(chat): fail unmet required client tool completions (#7463)

fix(combo): reject buffered required-tool false successes (#7741)

Partial fix on dev: #7741 (squash 816e84b) makes a buffered native combo child that skips a required tool_choice fail with required_tool_contract_unsatisfied instead of reporting success, and #7463 covers translated Chat routes with required_tool_not_called. Streaming native combo children are not covered yet, so this issue stays open for that half.

fix(responses): enforce required tools in native combo streams (#7831)

The streaming half is now on dev via #7831 (merge commit 8145d52). With #7741 (buffered native combo children) and #7463 (translated Chat), all three paths named in this issue now enforce tool_choice: "required" and named function choices:

A native SSE combo child that completes with no required call is not a combo success. If nothing client-visible was committed (only lifecycle or reasoning events, held for at most 1,000 ms), the combo moves to its next eligible target without leaking the discarded child's bytes. After text, a tool event, hosted-tool execution or any ambiguous output, the request ends with a typed required_tool_contract_unsatisfied failure instead of a false completed, with no replay.
The requirement comes from the caller's request. Authorized hosted substitutions recorded for the actual send (for example modelPreferHostedTools image generation) count as fulfilment. A requirement the outbound request cannot represent after capability filtering is not silently dropped.
Non-required requests, direct native requests, runTurn and WebSocket paths are unchanged. Tool catalogs (including the 151-tool case), call IDs and function_call_output continuation are covered by the regressions.

Not established here: live NVIDIA or other provider acceptance, and native Linux/macOS runtime. Coverage is fixture upstreams plus hosted Ubuntu CI. Closing; reopen with a request ID if a required-tool combo still reports completed without the call.

Navigation Menu

Repository navigation

[Bug] Responses combo treats completed text as success despite tool_choice=required (NVIDIA target) #7456

Client or integration

Area

Summary

Reproduction

Version

Operating system

Provider and model

Logs or error output

Screenshots and supporting files

Redacted configuration

Expected behavior / proposed acceptance

Privacy and provenance

Checks

Issue reopened

Additional live evidence: SSE/streaming is affected too (OpenCodex 2.89.0)

Additional implementation finding and tested mitigation (2026-10-21)

Production acceptance update, 2026-10-21 — streaming control passes with verified Sol first

New live differential: `required` fails while explicit named selection succeeds (2.89.0)

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[Bug] Responses combo treats completed text as success despite tool_choice=required (NVIDIA target) #7456

Description

Client or integration

Area

Summary

Reproduction

Version

Operating system

Provider and model

Logs or error output

Screenshots and supporting files

Redacted configuration

Expected behavior / proposed acceptance

Privacy and provenance

Checks

Activity

github-actions commented on Oct 20, 2026

github-actions commented on Oct 20, 2026

Issue reopened

RHODIZSECURITY commented on Oct 20, 2026

Additional live evidence: SSE/streaming is affected too (OpenCodex 2.89.0)

RHODIZSECURITY commented on Oct 21, 2026

Additional implementation finding and tested mitigation (2026-10-21)

RHODIZSECURITY commented on Oct 21, 2026

Production acceptance update, 2026-10-21 — streaming control passes with verified Sol first

Ingwannu commented on Oct 21, 2026

Ingwannu commented on Oct 21, 2026

RHODIZSECURITY commented on Oct 21, 2026

New live differential: required fails while explicit named selection succeeds (2.89.0)

lidge-jun commented on Oct 26, 2026

lidge-jun commented on Oct 28, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

New live differential: `required` fails while explicit named selection succeeds (2.89.0)