Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 37 additions & 1 deletion docs-site/src/content/docs/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,40 @@ fallback for upstream rate-limit codes `1302`/`1305` → the 60-second default.
`Retry-After: 0` remains an immediate upstream directive rather than being replaced by a configured
cooldown.

### Last-resort targets

A brief cooldown on a preferred target otherwise routes straight to whatever
comes next in the list — including a target you only ever wanted used in an
emergency. Mark it, and tell the combo to wait first:

```json
{
"strategy": "failover",
"cooldownWaitPolicy": "before-last-resort",
"waitForCooldownMs": 10000,
"targets": [
{ "provider": "provider-a", "model": "model-a" },
{ "provider": "provider-b", "model": "model-b" },
{ "provider": "provider-c", "model": "model-c", "lastResort": true }
]
}
```

With the policy set, selection tries the normal targets first. If they are only
cooling and the earliest cooldown expires inside `waitForCooldownMs`, the
request waits for that instead of dispatching the last-resort target.

**The policy only ever defers.** When no normal target can be reached — every
one cooling past the budget, already attempted, or ruled out — the last-resort
target is dispatched as usual. A policy that could withhold it would turn a
fallback into an outage, which is worse than the premature routing it prevents.
The same applies to a combo whose targets are *all* marked `lastResort`: it
dispatches normally.

`lastResort` is inert unless `cooldownWaitPolicy` is set, and both are omitted
by default, so existing combos are unaffected. Only the exact string
`before-last-resort` opts in.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

The current request never retries the same attempted target. Later requests skip a cooled target until its
cooldown expires; request-local compatibility rejections do not cool the target. A `Retry-After` HTTP-date that is already in the past is also preserved as an
immediate upstream directive, just like `Retry-After: 0`. Set `waitForCooldownMs` to allow a later
Expand Down Expand Up @@ -416,12 +450,14 @@ Combos are stored in the top-level `combos` object, keyed by combo id:

| Field | Required | Default | Rules |
| --- | --- | --- | --- |
| `targets` | Yes | — | Non-empty ordered array of configured `{ provider, model, weight? }` targets. Duplicate provider/model pairs are rejected. |
| `targets` | Yes | — | Non-empty ordered array of configured `{ provider, model, weight?, lastResort? }` targets. Duplicate provider/model pairs are rejected. |
| `targets[].weight` | No | `1` | Integer from 1 to 10,000. Used by round-robin and random; ignored by failover, least-used, and reset-window. |
| `targets[].lastResort` | No | `false` | Marks an emergency-only target. Inert unless `cooldownWaitPolicy` is set. Never makes a target permanently ineligible: when no normal target can be reached it is dispatched as usual. |
| `strategy` | No | `"failover"` | `"failover"`, `"round-robin"`, `"random"`, `"least-used"`, or `"reset-window"`. |
| `stickyLimit` | No | `1` | Integer from 1 to 100 successful requests per round-robin selection. Applies only to round-robin. |
| `cooldownMs` | No | unset → upstream fallback (5 s for request-rate 429 codes `1302`/`1305`, otherwise 60 s) | Integer from 1 to 600000. When set, applies as the per-target cooldown whenever no usable upstream `Retry-After` or Codex reset signal exists, including request-rate 429s; when unset, uses the upstream fallback. |
| `waitForCooldownMs` | No | `0` | Integer from 0 to 600000. Maximum time to wait for the earliest eligible cooling target before returning `combo_unavailable`; abort cancels the wait. |
| `cooldownWaitPolicy` | No | unset | `"before-last-resort"` defers targets marked `lastResort` while a normal target is cooling and that cooldown fits inside `waitForCooldownMs`. Only that exact string opts in. The deferral wait and the ordinary wait share one `waitForCooldownMs` budget per selection attempt. |
| `defaultEffort` | No | `null` | `low`, `medium`, `high`, `xhigh`, `max`, or `ultra`; resolved against each target's advertised ladder. |
| `defaultEffortMode` | No | `"fallback"` | `"fallback"` preserves explicit caller effort. `"force"` overrides valid caller effort, requires a valid non-null default, and can increase cost and latency. |
| `reasoningEffortMode` | No | `"strict"` | `"strict"` intersects every known target ladder, so one target advertising no effort control empties the combo's picker. `"adaptive"` excludes those empty ladders from the published intersection. At dispatch, explicit empty or adaptive unknown ladders remove unsupported effort/thinking controls while preserving supported non-effort reasoning fields such as `reasoning.summary`; known non-empty targets keep existing effort resolution. |
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/ja/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,10 +236,12 @@ ocx combo remove <id> --yes
| --- | --- | --- | --- |
| `targets` |はい | — |構成された `{ provider, model, weight? }` ターゲットの空でない順序付けされた配列。重複するプロバイダーとモデルのペアは拒否されます。 |
| `targets[].weight` |いいえ | `1` | 1 ~ 10,000 の整数。`round-robin` と `random` で使用され、`failover`、`least-used`、`reset-window` では無視されます。 |
| `targets[].lastResort` | いいえ | `false` | 緊急時専用のターゲットを示します。`cooldownWaitPolicy` を設定しない限り無効です。ターゲットを恒久的に除外することはありません。通常のターゲットに到達できない場合は通常どおりディスパッチされます。 |
| `strategy` |いいえ | `"failover"` | `"failover"`、`"round-robin"`、`"random"`、`"least-used"`、`"reset-window"`。 |
| `stickyLimit` |いいえ | `1` | `round-robin` の 1 回の選択あたり、成功したリクエスト数を指定する 1 ~ 100 の整数。`round-robin` にのみ適用されます。 |
| `cooldownMs` |いいえ | 未設定 → アップストリーム フォールバック(リクエストレート 429 コード `1302`/`1305` では 5 秒、それ以外では 60 秒) | 1 ~ 600000 の整数。設定時は、使用可能なアップストリーム `Retry-After` または Codex リセットシグナルがない場合に、リクエストレート 429 を含むターゲットごとのクールダウンとして適用されます。未設定時はアップストリーム フォールバックを使用します。 |
| `waitForCooldownMs` |いいえ | `0` | 0 ~ 600000 の整数。最も早く利用可能になる冷却中のターゲットを待ってから `combo_unavailable` を返すまでの最大待機時間。中止すると待機はキャンセルされます。 |
| `cooldownWaitPolicy` | いいえ | 未設定 | `"before-last-resort"` は、通常のターゲットがクールダウン中で、その残り時間が `waitForCooldownMs` に収まる間、`lastResort` を付けたターゲットを後回しにします。この文字列のみが有効です。後回しの待機と通常の待機は、1 回の選択につき同じ `waitForCooldownMs` の予算を共有します。 |
| `defaultEffort` |いいえ | `null` | `low`、`medium`、`high`、`xhigh`、`max`、または `ultra`;呼び出し元が努力を省略し、ターゲットがサポートをアドバタイズした場合にのみ適用されます。 |
| `reasoningEffortMode` | いいえ | `"strict"` | `strict` または `adaptive`。混在する capability の共通部分と対象別の制御正規化を選択します。 |
| `alias` |いいえ |なし |オプションのトリミングされたパブリック モデル ID。上記のエイリアス ルールを使用します。空の値はエイリアスなしで保存されます。 |
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/ko/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,10 +240,12 @@ ocx combo remove <id> --yes
| --- | --- | --- | --- |
| `targets` | 예 | — | 설정된 `{ provider, model, weight? }` 대상의 비어 있지 않은 순서가 있는 배열이어야 합니다. 중복된 provider/model 쌍은 거부됩니다. |
| `targets[].weight` | 아니요 | `1` | 1에서 10,000 사이의 정수입니다. `round-robin`과 `random`에서 사용되며, `failover`, `least-used`, `reset-window`에서는 무시됩니다. |
| `targets[].lastResort` | 아니요 | `false` | 비상용 대상임을 표시합니다. `cooldownWaitPolicy`를 설정하지 않으면 아무 효과가 없습니다. 대상을 영구히 제외하지는 않습니다. 일반 대상에 도달할 수 없으면 평소대로 디스패치됩니다. |
| `strategy` | 아니요 | `"failover"` | 허용되는 값은 `"failover"`, `"round-robin"`, `"random"`, `"least-used"`, `"reset-window"`입니다. |
| `stickyLimit` | 아니요 | `1` | 한 번의 `round-robin` 선택에 유지되는 성공 요청 수로, 1에서 100 사이의 정수입니다. `round-robin`에만 적용됩니다. |
| `cooldownMs` | 아니요 | 미설정 → 업스트림 폴백(요청 속도 제한 429 코드 `1302`/`1305`는 5초, 그 외는 60초) | 1에서 600000 사이의 정수입니다. 설정하면 사용 가능한 업스트림 `Retry-After` 또는 Codex 재설정 신호가 없을 때 요청 속도 제한 429를 포함한 대상별 쿨다운으로 적용됩니다. 설정하지 않으면 업스트림 폴백을 사용합니다. |
| `waitForCooldownMs` | 아니요 | `0` | 0에서 600000 사이의 정수입니다. `combo_unavailable`을 반환하기 전에 가장 먼저 적합해지는 쿨다운 중인 대상을 기다리는 최대 시간입니다. 중단하면 대기가 취소됩니다. |
| `cooldownWaitPolicy` | 아니요 | 미설정 | `"before-last-resort"`는 일반 대상이 쿨다운 중이고 그 잔여 시간이 `waitForCooldownMs` 안에 들어올 때 `lastResort` 대상을 뒤로 미룹니다. 이 문자열만 적용됩니다. 미루는 대기와 일반 대기는 선택 시도마다 같은 `waitForCooldownMs` 한도를 함께 씁니다. |
| `defaultEffort` | 아니요 | `null` | `low`, `medium`, `high`, `xhigh`, `max`, 또는 `ultra`입니다. 호출자가 effort를 생략하고 대상이 지원을 광고할 때만 적용됩니다. |
| `reasoningEffortMode` | 아니요 | `"strict"` | `strict` 또는 `adaptive`; 혼합 capability의 교집합과 대상별 제어 정규화를 선택합니다. |
| `alias` | 아니요 | 없음 | 선택적으로 앞뒤 공백을 제거한 공개 모델 ID입니다. 위의 alias 규칙을 따릅니다. 빈 값은 alias 없음으로 저장됩니다. |
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/ru/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,10 +289,12 @@ Combo хранятся в объекте верхнего уровня `combos`,
| --- | --- | --- | --- |
| `targets` | Yes | — | Непустой упорядоченный массив настроенных целей `{ provider, model, weight? }`. Дубли пар provider/model запрещены. |
| `targets[].weight` | No | `1` | Целое число от 1 до 10 000. Используется стратегиями `round-robin` и `random`; игнорируется стратегиями `failover`, `least-used` и `reset-window`. |
| `targets[].lastResort` | Нет | `false` | Помечает цель как резервную, только для аварийных случаев. Не действует, пока не задан `cooldownWaitPolicy`. Никогда не исключает цель навсегда: если ни одна обычная цель недоступна, она используется как обычно. |
| `strategy` | No | `"failover"` | `"failover"`, `"round-robin"`, `"random"`, `"least-used"` или `"reset-window"`. |
| `stickyLimit` | No | `1` | Целое число от 1 до 100 успешных запросов на один выбор `round-robin`. Применяется только к `round-robin`. |
| `cooldownMs` | No | не задано → fallback upstream (5 с для rate-limit 429 с кодами `1302`/`1305`, иначе 60 с) | Целое число от 1 до 600000. Если задано, применяется как cooldown каждой цели, когда нет пригодного upstream `Retry-After` или сигнала сброса Codex, включая rate-limit 429; если не задано, используется fallback upstream. |
| `waitForCooldownMs` | No | `0` | Целое число от 0 до 600000. Максимальное время ожидания самой ранней подходящей цели в cooldown перед возвратом `combo_unavailable`; отмена запроса отменяет ожидание. |
| `cooldownWaitPolicy` | Нет | не задан | `"before-last-resort"` откладывает цели с `lastResort`, пока обычная цель находится в остывании и это остывание укладывается в `waitForCooldownMs`. Включает только эта строка. Отложенное ожидание и обычное ожидание делят один бюджет `waitForCooldownMs` на одну попытку выбора. |
| `defaultEffort` | No | `null` | `low`, `medium`, `high`, `xhigh`, `max` или `ultra`; применяется только когда вызывающая сторона не указала effort, а цель объявляет поддержку. |
| `reasoningEffortMode` | Нет | `"strict"` | `strict` или `adaptive`; задаёт пересечение возможностей и нормализацию параметров конкретной цели. |
| `alias` | No | none | Необязательный обрезанный публичный id модели; используйте правила alias выше. Пустое значение хранится как отсутствие alias. |
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/zh-cn/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -265,10 +265,12 @@ combo 会存储在顶层的 `combos` 对象中,并以 combo id 作为键:
| --- | --- | --- | --- |
| `targets` | 是 | — | 非空、有顺序的数组,元素为已配置的 `{ provider, model, weight? }` 目标。重复的 provider/model 对会被拒绝。 |
| `targets[].weight` | 否 | `1` | 1 到 10,000 的整数。`round-robin` 和 `random` 会使用它;`failover`、`least-used` 和 `reset-window` 会忽略它。 |
| `targets[].lastResort` | 否 | `false` | 标记为仅在紧急情况下使用的目标。未设置 `cooldownWaitPolicy` 时不生效。它不会永久排除该目标:当没有普通目标可用时,仍会照常派发。 |
| `strategy` | 否 | `"failover"` | `"failover"`、`"round-robin"`、`"random"`、`"least-used"` 或 `"reset-window"`。 |
| `stickyLimit` | 否 | `1` | 每次 `round-robin` 选择可连续处理 1 到 100 个成功请求。仅适用于 `round-robin`。 |
| `cooldownMs` | 否 | 未设置 → 上游回退值(请求速率限制代码为 `1302`/`1305` 的 429 为 5 秒,否则为 60 秒) | 1 到 600000 的整数。设置后,只要没有可用的上游 `Retry-After` 或 Codex 重置信号,就会作为每个目标的冷却时间应用,包括请求速率限制 429;未设置时使用上游回退值。 |
| `waitForCooldownMs` | 否 | `0` | 0 到 600000 的整数。在返回 `combo_unavailable` 前等待最早恢复资格的冷却中目标的最长时间;请求中止会取消等待。 |
| `cooldownWaitPolicy` | 否 | 未设置 | `"before-last-resort"` 会在普通目标处于冷却中、且其剩余时间在 `waitForCooldownMs` 之内时,推迟使用标记为 `lastResort` 的目标。仅此字符串生效。推迟等待与普通等待在每次选择中共用同一个 `waitForCooldownMs` 预算。 |
| `defaultEffort` | 否 | `null` | `low`、`medium`、`high`、`xhigh`、`max` 或 `ultra`;仅当调用方省略 effort 且目标声明支持时才会应用。 |
| `reasoningEffortMode` | 否 | `"strict"` | `strict` 或 `adaptive`;选择混合能力交集和目标级控制归一化。 |
| `imageInput` | 否 | `"auto"` | `"auto"` 或 `"disabled"`。`"auto"` 仅在每个目标都支持图片时发布图片能力;`"disabled"` 强制仅文本(从对外能力中去掉图片,并在分发前拒绝带图请求)。 |
Expand Down
Loading
Loading