Skip to content

feat(models): declare capabilities on combo entries in /v1/models (#3901) - #3909

Open
ntdatt812 wants to merge 1 commit into
decolua:masterfrom
ntdatt812:fix/3901-combo-capabilities
Open

ntdatt812 wants to merge 1 commit into
decolua:masterfrom
ntdatt812:fix/3901-combo-capabilities

Conversation

@ntdatt812

Copy link
Copy Markdown
Contributor

Closes #3901.

Every single-provider entry in GET /v1/models declares a capabilities object. A combo declared none at all:

{"id":"gpt-5.6-sol","object":"model","owned_by":"combo"}

Codex CLI reads that field to decide what tool schema to send on the actual call. Seeing no declared tool support it downgrades, and the reporter's requestDetails shows what reaches the provider: two stub tools with empty parameters.properties — nothing the model can call, and no error anywhere in the chain. A combo looks like a model that cannot use tools, and the failure reads as the model being bad at its job.

What it declares

Only what every member honours. reorderModelsByCapability sorts a combo's members by fit but never drops one — the comment on it says so, "never drops a model (fallback intact)" — so a request can still land on any member. Declaring tools: true because the first member supports them is a promise fallback breaks silently, on the turn a tool is actually needed. The conservative AND is the only answer the pool can keep.

That is safe to compute because getCapabilitiesForModel always answers: it merges with DEFAULT_CAPABILITIES, so an unknown model reports the default (tools: true) rather than nothing. There is no "unknown" here to mistake for false.

Only the flags a client reads to decide what to send — tools, vision, reasoning, search, and the input/output modality flags. thinkingFormat, thinkingRange, contextWindow and maxOutput describe how to talk to one specific model; merged across a pool they would be wrong rather than merely incomplete, so they are deliberately left off.

Verification

npx vitest@3 run --config tests/vitest.config.js tests/unit/combo-capabilities-3901.test.js
→ 6 passed

npx next build — 0 errors.

Checked by breaking it: changing the AND to an OR (one member is enough) fails 3 of the 6 — the withheld-feature case, the missing-record case, and the per-flag independence case. That is the mutation that matters here, because OR is the reading that makes the reporter's symptom go away while reintroducing the silent-failure version of it.

One test exists only to pin that an absent capabilities is still possible for a combo with genuinely no members, and never as a quiet fallback for one that has them — an absent field is what the bug looked like, so it must not become the error path.

Note on overlap

This touches the same else branch as my #3880, which adds context_length / max_completion_tokens to combo entries by the same "smallest member" reasoning. They are independent and I have kept this one standalone against master so it can be taken on its own; if #3880 goes first this rebases to one hunk, and I am happy to do that.

…colua#3901)

Every single-provider entry in the catalog declares a `capabilities`
object. A combo declared none at all -- just { id, object, owned_by }.

Codex CLI reads that field to decide what tool schema to send. Seeing no
declared tool support it downgraded the schema on the actual /v1/responses
call, and the model was handed two stub tools with empty
parameters.properties: nothing it could call, and no error anywhere.

A request routed to a combo may land on any member -- reordering by
capability fit sorts but never drops -- so the pool can only promise what
every member honours. Declaring tools:true because one member has them is
a promise fallback breaks silently, on the turn it matters.

Only the flags a client reads to decide what to send are reported. Thinking
wire format, budget ranges and token limits describe how to talk to one
specific model and would be wrong rather than merely incomplete if merged
across a pool.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Combo models silently drop function-calling for the downstream client (Codex CLI degrades to two empty stub tools)

1 participant