Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/server/chat-completions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -341,7 +341,8 @@ async function handleChatCompletionsWithBudget(
} else if (internalBody.store === undefined) {
internalBody.store = false;
}
if (settledRoute && internalBody.reasoning !== undefined) {
if (settledRoute && !settledRoute.combo && settledRoute.routeKind !== "policy"
&& internalBody.reasoning !== undefined) {
Comment thread
Ingwannu marked this conversation as resolved.
const { stripEmptyLadderEffort, supportedLadderFor } = await import("./effort-policy");
const ladder = supportedLadderFor({ provider: settledRoute.provider, modelId: settledRoute.modelId });
const next = stripEmptyLadderEffort(internalBody.reasoning, ladder);
Expand Down
14 changes: 13 additions & 1 deletion src/server/responses/core-normalize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import { subagentFallbackNeedsModelEntitlements } from "../../codex/subagent-mod
import { MAIN_CODEX_ACCOUNT_ID } from "../../codex/main-account";
import type { RequestLogContext } from "../request-log";
import type { HandleResponsesOptions } from "./core-options";
import { prepareEffortNormalization } from "../effort-policy";
import { prepareEffortNormalization, stripEmptyLadderEffort, supportedLadderFor } from "../effort-policy";
import { resolveOpenCodeGoTransport } from "../../providers/opencode-go-transport";
import { getOrAllocateRequestSessionLane } from "../request-log-conversation";
import { shouldPreparePlaintextV2AgentMessages } from "../../responses/plaintext-v2-agent-messages";
Expand Down Expand Up @@ -307,6 +307,18 @@ export async function applyFinalRouteRequestNormalization(args: {
logCtx.requestedEffort = `${logCtx.requestedEffort ?? "max"}->${clamped}`;
}
}
// Chat ingress cannot strip effort against a provisional policy pick. Apply the
// concrete target's restriction to BOTH adapter options and the raw wire copy;
// policy-fallback retains the original body before this attempt-local mutation.
if (inboundWire === "chat" && supportedLadderFor(route)?.length === 0) {
parsed.options.reasoning = undefined;
const raw = parsed._rawBody as { reasoning?: unknown } | undefined;
if (raw) {
const reasoning = stripEmptyLadderEffort(raw.reasoning, []);
if (reasoning === undefined) delete raw.reasoning;
else raw.reasoning = reasoning;
}
}
recordAttemptRequestedEffort(logCtx);
logCtx.modelSupportsServiceTier = SERVICE_TIER_ADAPTERS.has(route.provider.adapter)
? modelServiceTierSupport
Expand Down
12 changes: 12 additions & 0 deletions structure/decisions/ADR-0110-chat-reasoning-failover-intent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# ADR-0110 — decision recorded under "Responses Failover"

- Contract owner: [transports/responses-failover.md](../transports/responses-failover.md)

## Decision record

- 목적과 의도: Preserve a Chat caller's reasoning effort across combo and policy failover while still omitting unsupported controls from each concrete upstream.
- 기존 구현 및 제약 조건: Chat ingress translated one shared Responses body, then stripped effort using the provisional settled route. Combo and policy dispatch already clone and normalize that body per target with the target's own reasoning ladder.
- 검토한 주요 대안: Keep ingress stripping; restore effort from the original Chat body after each failure; attach a second private metadata field; or skip ingress stripping for unresolved routes and use the existing per-target normalizer.
- 선택한 방식: Apply ingress empty-ladder stripping only to a non-combo, non-policy concrete route. Combo and policy retain the translated reasoning object and normalize an isolated child body for every attempt.
- 다른 대안 대신 이 방식을 선택한 이유: The shared body is caller intent, not an attempt wire shape. Reconstructing after mutation is lossy and a second metadata channel can drift, while the existing child normalizer already owns target capability mapping.
- 장점, 단점 및 영향: A capable fallback receives the requested effort even when an earlier target has an empty ladder; unsupported targets still omit it. The unresolved body retains the small reasoning object until a target is chosen.
9 changes: 9 additions & 0 deletions structure/transports/responses-failover.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,15 @@ Native Chat applies qualifying effort ceilings independently of model pins; pin

Pool quota producers and account commands follow the [bounded raw-observation contract](../providers/openai-accounts.md#bounded-pool-quota-observations), separate from the latest display snapshot and capacity estimates; account quota surfaces use [safe probe diagnostics](inventory.md#account-quota-failure-diagnostics) separately from quota validity, credential health and routing authority. Raw-byte readers on this path supply their own byte and deadline budgets under the [bounded ingestion contract](inventory.md#bounded-response-ingestion-and-orcarouter-login).

Translated Chat requests preserve caller reasoning intent until a combo or policy selects a
concrete target. Empty-ladder stripping and effort mapping apply to each attempt copy, never the
shared ingress body, so a later capable fallback still receives the caller's requested effort.
`src/server/responses/core-normalize.ts` strips an empty ladder from both parsed adapter options
and raw reasoning on each translated Chat attempt, preserving summary controls. Policy fallback
captures the original body before this normalization, including for its first candidate.

> Decision record: [ADR-0110](../decisions/ADR-0110-chat-reasoning-failover-intent.md)

Live sideband admission and its bounded upstream handshake follow the [runtime contract](../runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.

Translated Chat request construction uses the [inline-image budget](streaming-health.md#translated-chat-inline-image-budget); the shared normalizer counts retained bytes even when a wire-specific drop callback keeps the image attached, rejects inputs above the safe decoded-pixel ceiling, caps native decode work process-wide, and stops queued work when the request is cancelled.
Expand Down
54 changes: 54 additions & 0 deletions tests/responses/chat-native-combo.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,60 @@ describe("native Chat candidates in a combo", () => {
});
});

test("Chat reasoning intent survives an empty-ladder first target and reaches failover", async () => {
const a = upstream(() => Response.json({ error: { message: "fixture outage" } }, { status: 503 }));
const b = upstream(() => responsesStream("reasoned fallback"));
const config = comboConfig(
{
a: provider("openai-responses", a.baseUrl, { reasoningEfforts: [] }),
b: provider("openai-responses", b.baseUrl, { reasoningEfforts: ["low", "high"] }),
},
[{ provider: "a", model: "m1" }, { provider: "b", model: "m2" }],
);

const { response, text } = await send(config, { stream: false, reasoning_effort: "high" });

expect(response.status).toBe(200);
expect(text).toContain("reasoned fallback");
expect(a.bodies).toHaveLength(3);
for (const body of a.bodies) {
expect((body.reasoning as Rec | undefined)?.effort).toBeUndefined();
}
expect(b.bodies).toHaveLength(1);
expect((b.bodies[0]!.reasoning as Rec | undefined)?.effort).toBe("high");
});

test("Chat policy fallback strips only the empty-ladder attempt's reasoning effort", async () => {
const a = upstream(() => Response.json({ error: { message: "fixture outage" } }, { status: 503 }));
const b = upstream(() => responsesStream("reasoned policy fallback"));
const config: OcxConfig = {
port: 0,
defaultProvider: "a",
providers: {
// Anthropic consumes parsed options rather than the Responses raw-body
// sanitizer, so this catches an effort that leaks past policy selection.
a: provider("anthropic", a.baseUrl, { models: ["m1"], reasoningEfforts: [] }),
b: provider("openai-responses", b.baseUrl, { models: ["m2"], reasoningEfforts: ["low", "high"] }),
},
routingProfiles: { daily: { candidates: [{ provider: "a", model: "m1" }, { provider: "b", model: "m2" }] } },
};

const { response, text, rows } = await send(config, {
model: "policy/daily", stream: false, reasoning_effort: "high", include_reasoning: true,
});

expect(response.status).toBe(200);
expect(text).toContain("reasoned policy fallback");
expect(a.bodies.length).toBeGreaterThan(0);
for (const body of a.bodies) {
expect(body.thinking).toBeUndefined();
expect(body.output_config).toBeUndefined();
}
expect(b.bodies).toHaveLength(1);
expect(b.bodies[0]!.reasoning).toMatchObject({ effort: "high", summary: "auto" });
expect(rows[0]!.attempts?.map(attempt => attempt.status)).toEqual([503, 200]);
});

test("a streamed native answer that fails after output is not re-sent to the next target", async () => {
const a = upstream(() => chatErrorStream("fixture broke mid-stream", "partial answer"));
const b = upstream(() => responsesStream("must not run"));
Expand Down
29 changes: 29 additions & 0 deletions tests/responses/reasoning-effort-summary-default.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,4 +110,33 @@ describe("reasoning effort preserves visible thinking when summary is omitted",
const child = concreteComboRequestBody(body, target, "high", ["high"]);
expect(child.reasoning).toEqual({ effort: "high", summary: "auto" });
});

test("Chat final-route normalization strips both shapes without losing summary or later-route effort", async () => {
const config: OcxConfig = {
port: 0,
defaultProvider: "a",
providers: {
a: { adapter: "anthropic", baseUrl: "https://a.example.test/v1", reasoningEfforts: [] },
b: { adapter: "anthropic", baseUrl: "https://b.example.test/v1", reasoningEfforts: ["low", "high"] },
},
};
for (const [provider, expectedEffort] of [["a", undefined], ["b", "high"]] as const) {
// Policy fallback parses each attempt from its preserved wire snapshot. Keep these
// cases independent so the empty-ladder mutation cannot manufacture the capable result.
const parsed = parseRequest({
model: `${provider}/test-model`, input: [], reasoning: { effort: "high", summary: "auto" },
});
const route = routeModel(config, `${provider}/test-model`);
await applyFinalRouteRequestNormalization({
Comment thread
Ingwannu marked this conversation as resolved.
parsed, route, config,
req: new Request("http://localhost/v1/responses"),
logCtx: { model: parsed.modelId, provider },
inboundWire: "chat",
});
expect(parsed.options.reasoning).toBe(expectedEffort);
expect((parsed._rawBody as { reasoning: { effort?: string; summary: string } }).reasoning)
.toEqual(expectedEffort ? { effort: expectedEffort, summary: "auto" } : { summary: "auto" });
expect(parsed.options.hideThinkingSummary).toBeUndefined();
}
});
});
Loading