Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs-site/src/content/docs/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,10 @@ native routing and catalog identity on the next sync.

For Responses requests that require a tool, Meta Responses targets are skipped before sending
because Meta does not support mandatory `tool_choice`. An eligible later target receives the
original choice. If every target is incompatible for this reason, the combo reports the constraint.
original choice, subject to its existing adapter compatibility rules. This includes hosted choices
such as `web_search`, `tool_search`, `computer_use` and `image_gen`, and MCP choices with a
nonempty `server_label` and an omitted, null or nonempty name, including inside required allowed-tools lists. Chat translation preserves a forced `tool_search` as the synthetic function choice. If every target is incompatible for this reason,
the combo reports the constraint. Chat targets that cannot preserve a mandatory selector, including models configured for auto-only tool choice, are also skipped before sending. Registered OpenAI API virtual models use the actual wire model's adapter and tool-choice policy for this decision. Malformed or unknown choices retain the provider's ordinary error path.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

`failover` selects the first eligible target in configuration order. A target is eligible when its
provider exists, is enabled, is not cooling down, and can handle any special request constraint.
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/ja/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ ocx combo show main

## 戦略を選択してください

必須の `tool_choice` を維持できないターゲットは送信前にスキップされ、すべて非対応の場合はその制約を示すエラーを返します。自動選択のみを許可する Chat モデルと、OpenAI 仮想モデルの実際の送信モデルのポリシーも判定対象です。直接の hosted/MCP 選択、`tool_search`、無効な選択値の扱いは[英語の基準説明](/guides/combos/#failover-ordered-primary-and-backups)を参照してください。

### フェイルオーバー: 順序付けされたプライマリとバックアップ

`failover` は、構成順序で最初の適格なターゲットを選択します。ターゲットは、プロバイダーが存在し、有効になっていて、クールダウン中でなく、特別なリクエスト制約を処理できる場合に適格となります。重みと `stickyLimit` はこの戦略に影響しません。
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/ko/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ alias는 클라이언트가 요청하는 공개 이름만 바꿉니다. 콤보

## 전략 선택

필수 `tool_choice`를 보존할 수 없는 대상은 전송 전에 건너뛰며, 모든 대상이 호환되지 않으면 해당 제약을 오류로 반환합니다. 자동 선택만 허용하는 Chat 모델과 OpenAI 가상 모델의 실제 전송 모델 정책도 이 판단에 반영됩니다. 직접 hosted/MCP 선택, `tool_search`, 잘못된 선택값의 처리 범위는 [영문 기준 설명](/guides/combos/#failover-ordered-primary-and-backups)을 참고하세요.

### 페일오버: 순서가 있는 기본값과 예비값

`failover`는 설정 순서에서 가장 먼저 적합한 대상을 선택합니다. 대상이 적합하려면 해당 공급자가 존재하고, 활성화되어 있고, 쿨다운 중이 아니며, 요청에 붙은 특수 조건을 처리할 수 있어야 합니다. 이 전략에서는 가중치와 `stickyLimit`이 영향을 주지 않습니다.
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/ru/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,8 @@ Alias меняет публичное имя, которое запрашива

## Выбор стратегии

Цель, которая не может сохранить обязательный `tool_choice`, пропускается до отправки; если совместимых целей нет, возвращается ошибка с этим ограничением. Проверка учитывает модели Chat с выбором только `auto` и политику фактически отправляемой модели для виртуальных моделей OpenAI. Правила для прямого выбора hosted/MCP, `tool_search` и некорректных значений приведены в [каноническом руководстве на английском](/guides/combos/#failover-ordered-primary-and-backups).

### Failover: основной вариант и резерв в заданном порядке

`failover` выбирает первую подходящую цель в порядке конфигурации. Цель считается подходящей,
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/zh-cn/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@ modality 和 reasoning metadata;目标显式声明的限制仍然优先,因

## 选择策略

无法保留强制 `tool_choice` 的目标会在发送前跳过;如果所有目标都不兼容,则返回说明该限制的错误。判断会考虑仅支持 `auto` 选择的 Chat 模型,以及 OpenAI 虚拟模型实际发送到上游的模型策略。直接 hosted/MCP 选择、`tool_search` 和无效选择值的处理范围请参阅[英文基准说明](/guides/combos/#failover-ordered-primary-and-backups)。

### 故障切换:按顺序的主目标和备用目标

`failover` 会按配置顺序选择第一个合格目标。当 provider 存在、已启用、未处于冷却中,并且能够满足任何特殊请求约束时,该目标就是合格的。权重和 `stickyLimit` 不影响这种策略。
Expand Down
1 change: 1 addition & 0 deletions src/responses/parser-tools.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ export function mapToolChoice(value: unknown): OcxRequestOptions["toolChoice"] {
if (t === "image_generation" || t === "image_gen") {
return { name: IMAGE_GEN_TOOL_NAME };
}
if (t === "tool_search") return { name: "tool_search" };
if (t === "allowed_tools" && Array.isArray(value.tools)) {
const names = value.tools
.map(allowedToolName)
Expand Down
7 changes: 5 additions & 2 deletions src/responses/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -162,12 +162,15 @@ const builtinToolSchema = z.object({ type: z.string() }).loose();
*/
export const HOSTED_TOOL_TYPES = [
"web_search", "web_search_preview", "file_search", "computer_use_preview",
"code_interpreter", "image_generation", "mcp",
"code_interpreter", "image_generation", "mcp", "tool_search", "computer_use", "image_gen",
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
] as const;

const hostedToolType = z.enum(HOSTED_TOOL_TYPES);

const allowedToolEntrySchema = z.object({ type: z.string(), name: z.string().optional() });
const allowedToolEntrySchema = z.union([
z.object({ type: z.literal("mcp"), server_label: z.string().min(1), name: z.string().nullable().optional() }),
z.object({ type: z.string(), name: z.string().optional() }),
]);

export const toolChoiceSchema = z.union([
z.literal("auto"),
Expand Down
36 changes: 24 additions & 12 deletions src/server/responses/combo-required-tool-contract.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
import { REQUIRED_TOOL_CONTRACT_CODE } from "../../lib/errors";
import { mapToolChoice } from "../../responses/parser-tools";
import { modelInList, type OcxProviderConfig } from "../../types";
import { hasResponseMediaType } from "../../lib/http-response-semantics";
import { applyReplayRefusalClientHeaders, isNonReplayableResponse, markResponseNonReplayable } from "../../lib/upstream-retry";
import { clientWireOf } from "../inference/client-wire";
Expand All @@ -15,26 +17,36 @@ const allowedHostedSelectors = new Set([
"code_interpreter", "image_generation", "image_gen", "tool_search",
]);

function knownToolSelector(value: unknown): boolean {
if (!value || typeof value !== "object" || Array.isArray(value)) return false;
const entry = value as Record<string, unknown>;
if (entry.type === "mcp") return typeof entry.server_label === "string" && !!entry.server_label.trim()
&& (entry.name === undefined || entry.name === null || (typeof entry.name === "string" && !!entry.name.trim()));
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
if (entry.name !== undefined && (typeof entry.name !== "string" || !entry.name.trim())) return false;
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
if (entry.type === "function" || entry.type === "custom") return typeof entry.name === "string";
return typeof entry.type === "string" && allowedHostedSelectors.has(entry.type);
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
}

/** Only definite, well-formed mandatory selectors justify excluding a destination. */
export function hasMandatoryToolChoice(rawBody: unknown): boolean {
if (!rawBody || typeof rawBody !== "object" || Array.isArray(rawBody)) return false;
const choice = (rawBody as Record<string, unknown>).tool_choice;
if (choice === "required") return true;
if (!choice || typeof choice !== "object" || Array.isArray(choice)) return false;
const selector = choice as Record<string, unknown>;
if (selector.type === "function" || selector.type === "custom") {
return typeof selector.name === "string" && selector.name.trim().length > 0;
}
return selector.type === "allowed_tools" && selector.mode === "required"
// Eligibility is broader than terminal inspection: hosted selectors retain their own terminal owner.
if (selector.type !== "allowed_tools") return knownToolSelector(selector);
return selector.mode === "required"
&& Array.isArray(selector.tools) && selector.tools.length > 0
&& selector.tools.every((tool: unknown) => {
if (!tool || typeof tool !== "object" || Array.isArray(tool)) return false;
const entry = tool as Record<string, unknown>;
if (entry.name !== undefined && (typeof entry.name !== "string" || !entry.name.trim())) return false;
if (entry.type === "function" || entry.type === "custom") return typeof entry.name === "string";
if (entry.type === "mcp") return typeof entry.server_label === "string" && !!entry.server_label.trim();
return typeof entry.type === "string" && allowedHostedSelectors.has(entry.type);
});
&& selector.tools.every(knownToolSelector);
}

/** A translated Chat target must not turn a mandatory selector into auto or none. */
export function chatCanPreserveMandatoryToolChoice(rawBody: unknown, provider: OcxProviderConfig, modelId: string): boolean {
if (!hasMandatoryToolChoice(rawBody)) return true;
if (modelInList(provider.autoToolChoiceOnlyModels, modelId)) return false;
const mapped = mapToolChoice((rawBody as Record<string, unknown>).tool_choice);
return mapped === "required" || (!!mapped && typeof mapped === "object");
}

const hostedCalls = new Map([
Expand Down
20 changes: 12 additions & 8 deletions src/server/responses/core-combo.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
import { effectiveAzureOutputBudget } from "../../adapters/azure";
import { buildResponsesTransport } from "../../adapters/openai-responses/transport";
import { isMetaAiResponsesDestination } from "../../responses/muse-tool-name-alias";
import { hasMandatoryToolChoice } from "./combo-required-tool-contract";
import { chatCanPreserveMandatoryToolChoice, hasMandatoryToolChoice } from "./combo-required-tool-contract";
import { isCodexAccountCooldownResponse } from "../../codex/account-cooldown-response";
import { resolveWireProtocolOverride } from "../adapter-resolve";
import { parseSyntheticRowId } from "../fast-row";
import { resolveOpenAiVirtualModel } from "../../providers/openai-virtual-models";
import { comboCooldownRetryAfterSeconds, isDefiniteContextOverflow, type CodexAccountModelRefusal } from "../../combos/failover";
import { fallbackTriggerForStatus, guardFallbackHop } from "../../routing/fallback-policy";
import {
Expand Down Expand Up @@ -539,13 +541,15 @@ export async function executeComboResponses(
const museToolEligible = (target: (typeof combo.targets)[number]): boolean => {
if (!hasMandatoryToolChoice(body)) return true;
try {
const route = routeConcreteModel(config, `${target.provider}/${target.model}`);
const selector = `${target.provider}/${target.model}`, { fastRow, effortRow } = parseSyntheticRowId(selector, config);
const route = routeConcreteModel(config, fastRow?.baseId ?? effortRow?.baseId ?? selector);
const modelId = resolveOpenAiVirtualModel(route.providerName, route.modelId)?.wireModelId ?? route.modelId;
const inboundWire = options.inboundWire ?? "responses";
const policy = captureRouteStaticPolicy(route.providerName, route.modelId, route.provider,
const policy = captureRouteStaticPolicy(route.providerName, modelId, route.provider,
route.staticPolicy.effectiveAlias, inboundWire);
const provider = resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire, policy);
return provider.adapter !== "openai-responses"
|| !isMetaAiResponsesDestination(buildResponsesTransport(provider, { headers: req.headers }).url);
const provider = resolveWireProtocolOverride(route.providerName, modelId, route.provider, inboundWire, policy);
return provider.adapter === "openai-chat" ? chatCanPreserveMandatoryToolChoice(body, provider, modelId)
: provider.adapter !== "openai-responses" || !isMetaAiResponsesDestination(buildResponsesTransport(provider, { headers: req.headers }).url);
} catch {
return true;
}
Expand Down Expand Up @@ -674,7 +678,7 @@ export async function executeComboResponses(
{ parentThreadId: inboundClientThreadId },
);
if (!options.abortSignal?.aborted && onlyMuseToolIncompatible()) {
return comboUnavailableResponse(`No available targets for combo: ${comboId}; Meta Responses does not support mandatory tool_choice`);
return comboUnavailableResponse(`No available targets for combo: ${comboId}; targets cannot preserve mandatory tool_choice`);
}
return options.abortSignal?.aborted
? clientCancelledResponse()
Expand All @@ -694,7 +698,7 @@ export async function executeComboResponses(
if (protocolRefusal) return protocolRefusal;
if (onlyReplayIncompatibleTargetsRemain()) return targetIncompatibleResponse();
if (!options.abortSignal?.aborted && onlyMuseToolIncompatible()) {
return comboUnavailableResponse(`No available targets for combo: ${comboId}; Meta Responses does not support mandatory tool_choice`);
return comboUnavailableResponse(`No available targets for combo: ${comboId}; targets cannot preserve mandatory tool_choice`);
}
return options.abortSignal?.aborted
? clientCancelledResponse()
Expand Down
2 changes: 1 addition & 1 deletion structure/providers-and-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -555,7 +555,7 @@ uses `src/adapters/openai-responses/muse-tool-choice.ts` to normalize `tool_choi
`additional_tools` items from `input`, and omits `tool_choice` and `parallel_tool_calls` from the
outgoing body. Forced, named, and `allowed_tools` selections fail with HTTP 400 before the
upstream send because Muse supports only `auto`. Filtering a required tool never changes the caller's obligation into `none` or `auto`. This rule applies only to the Meta Responses
destination. The input body, historical tool calls and results, and non-Meta requests keep their existing meaning.
destination. Combo eligibility excludes it before sending for known mandatory selectors, including direct hosted types and MCP selectors with a nonempty server label; malformed and unknown selectors retain the adapter error path. The input body, historical tool calls and results, and non-Meta requests keep their existing meaning.

## Shared type declarations

Expand Down
8 changes: 6 additions & 2 deletions structure/transports/responses-combo-boundaries.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,15 @@ and `tests/responses/responses-combo-exhausted-error.test.ts`.
## Buffered required-tool contract

Before selection, `src/server/responses/core-combo.ts` excludes resolved Meta Responses destinations
for well-formed mandatory `tool_choice` selectors, including required allowed-tools lists. Normal and
for well-formed mandatory `tool_choice` selectors, including direct hosted/MCP choices and required allowed-tools lists. MCP's optional name may be null; an invalid name or server label still abstains. Normal and
recovery picks share the check, resolving the inbound wire's static model policy and actual Responses
transport URL; malformed and unknown selectors retain their ordinary error path. The next target
receives the original selector, and an all-Meta exhaustion names the constraint before any send.
Coverage: `tests/server/server-combo-required-tool-contract.test.ts`.
The child parser accepts `tool_search`, `computer_use` and `image_gen` as recognized hosted choices.
Its raw body preserves the selector; existing destination compatibility, including routed private
tool-search lowering, still applies after parsing. Chat translation maps a direct `tool_search`
choice to its existing synthetic function name. MCP null names also parse inside required allowed-tools lists; function/custom null names remain invalid. A Chat destination is skipped before send when translation would degrade a mandatory choice to auto/none, including the resolved model's auto-only tool-choice policy; exhaustion reports the tool-choice constraint. Eligibility resolves synthetic Fast/effort selectors before model policy, matching child request preparation while preserving exact configured IDs.
Registered OpenAI API virtual models resolve to their wire model before adapter and auto-only policy checks, using the same registry resolver as child normalization. Other providers and unregistered IDs remain literal. Coverage: `tests/server/server-combo-required-tool-contract.test.ts`.

Before combo success bookkeeping, native delivery in `src/server/responses/passthrough-delivery.ts`
uses `src/server/responses/combo-required-tool-contract.ts` to check successful native
Expand Down
23 changes: 23 additions & 0 deletions tests/responses/responses-parser.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,29 @@ import { repoPath } from "../helpers/repo-root";
import { normalizeThinkingOptions, resolveThinkingEnabled } from "../../src/clients/reasoning-controls";
import type { OcxProviderConfig } from "../../src/types";

test.each(["tool_search", "computer_use", "image_gen", "mcp"])("preserves direct hosted selector %s in the raw Responses body", type => {
const choice = type === "mcp" ? { type, server_label: "docs", name: null } : { type };
const raw = { model: "fixture", input: "hello", tool_choice: choice };
expect(parseRequest(raw)._rawBody?.tool_choice).toEqual(choice);
});

test("translated tool_search keeps the mandatory synthetic function choice", () => {
const parsed = parseRequest({ model: "fixture", input: "hello", tools: [{ type: "tool_search" }],
tool_choice: { type: "tool_search" } });
expect(parsed.options.toolChoice).toEqual({ name: "tool_search" });
expect(parsed.context.tools?.map(tool => tool.name)).toContain("tool_search");
});

test("MCP allowed-tools null name preserves its raw selector without accepting function null names", () => {
const choice = { type: "allowed_tools", mode: "required", tools: [{ type: "mcp", server_label: "docs", name: null }] };
expect(parseRequest({ model: "fixture", input: "hello", tool_choice: choice })._rawBody?.tool_choice).toEqual(choice);
for (const type of ["function", "custom"]) {
expect(() => parseRequest({ model: "fixture", input: "hello", tool_choice: {
...choice, tools: [{ type, name: null }],
} })).toThrow();
}
});

describe("generic effort-only client compatibility", () => {
const provider: OcxProviderConfig = {
adapter: "openai-chat", baseUrl: "https://fixture.invalid/v1",
Expand Down
Loading
Loading