Skip to content

command-code: shipped effort ladders are narrower than the live API accepts (and config cannot widen a pinned row) #5096

Closed

Description

@hanlin-luo

Client or integration

Codex CLI

Provider or upstream service

command-code (Command Code Provider API)

OpenCodex version

2.59.0

Endpoint or capability

reasoning metadata (COMMAND_CODE_MODEL_REASONING_EFFORTS in src/providers/command-code-efforts.ts)

Current behaviour

The shipped table pins several Command Code models to effort ladders narrower than the live API accepts, and 38 live models have no row at all. Because applyDirectReasoningEffortContracts() re-applies the registry value for those models (and the provider block is re-derived from the table on every ocx service restart), a user cannot widen a pinned ladder from config.json — ocx config set providers.command-code.modelReasoningEfforts (49 entries) is back to 17 entries after a restart.

Observed downstream: command-code/deepseek-deepseek-v4.1-flash advertises high, max, ultra in ~/.codex/opencodex-catalog.json / models_cache.json, so Codex offers no low, medium, or xhigh for a model the gateway serves them on. Models with no row advertise no effort control at all.

Expected behaviour

The table's rows should match what the upstream accepts. Measured 2026-09-17 against POST https://api.commandcode.ai/provider/v1/chat/completions (GOAT plan key, OpenAI-completions shape):

  • reasoning_effort accepts exactly low | medium | high | xhigh | max.
  • none and minimal are rejected for every model: 400 invalid_request_error: Invalid option: expected one of "low"|"medium"|"high"|"xhigh"|"max".
  • Omitting the field still returns reasoning / reasoning_details, so there is no honest "off" rung.
  • The accepted subset varies per model and is not contiguous (moonshotai/Kimi-K2.7-Code-Highspeed has no medium; poolside/laguna-s-2.1-free accepts only medium).

Rows that disagree with the measurement:

model shipped measured
deepseek/deepseek-v4.1-flash high, max low, medium, high, xhigh, max
deepseek/deepseek-v4-flash high, max low, medium, high, xhigh, max
deepseek/deepseek-v4-flash-vision-exp high, max low, medium, high, xhigh, max
z-ai/glm-5.3-flash low, high, max low, medium, high, xhigh, max
zai-org/GLM-5.3 low, high, max low, medium, high, xhigh, max
Qwen/Qwen3.8-Flash low, medium, high, max low, medium, high, xhigh, max
google/gemini-3.7-flash low, medium, high low, medium, high, xhigh, max

Missing rows (38 live models), all measured: moonshotai/Kimi-K3 low..max, moonshotai/Kimi-K2.7-Code low..xhigh (no max), moonshotai/Kimi-K2.7-Code-Highspeed low, high, xhigh, max, MiniMaxAI/MiniMax-M3 low..max, xiaomi/mimo-v2.5 low..max, xiaomi/mimo-v2.5-pro low..high, xai/grok-4.5 / xai/grok-4.6 low..max, tencent/hy3-paid / tencent/hy4-preview low..max, stepfun/Step-3.7-Flash low..max, Qwen/Qwen3.8-Max low..max, Qwen/Qwen3.8-27B low..max, Qwen/Qwen3.7-* low..xhigh, Qwen/Qwen3.6-* low..xhigh, meta/muse-spark-1.2* / 1.3* low..max, nvidia/nemotron-3-ultra-550b-a55b low..max, meituan/LongCat-2.0:free low..max, inclusionai/ling-3.0-flash-sante:free low..max, thinkingmachines/inkling-small low..max, poolside/laguna-s-2.1-free medium only, google/gemini-3.8-flash low..high.

Minimal redacted request or reproduction

# every rung, per model
curl -sS https://api.commandcode.ai/provider/v1/chat/completions \
  -H "Authorization: Bearer $CMD_KEY" -H 'Content-Type: application/json' \
  -d '{"model":"deepseek/deepseek-v4.1-flash","messages":[{"role":"user","content":"hi"}],
       "max_tokens":64,"reasoning_effort":"xhigh"}'

# what the table produces today (Codex catalog), after `ocx sync`:
jq -r '.models[] | select(.slug=="command-code/deepseek-deepseek-v4.1-flash")
       | [.supported_reasoning_levels[].effort] | join(",")' ~/.codex/opencodex-catalog.json

Two probe traps worth knowing: max_tokens: 8 makes some models answer Invalid 'max_output_tokens' (looks like an unsupported model), and an 8×8 test PNG is rejected as invalid image by several upstreams (looks like "no vision"). Use max_tokens >= 64 and a real PNG.

Actual response or error

# reasoning_effort from the shipped rows that is not accepted / not offered:
$ curl ... -d '{"model":"deepseek/deepseek-v4.1-flash", ..., "reasoning_effort":"xhigh"}'
HTTP 200  (accepted — but Codex never offers it)

$ curl ... -d '{"model":"deepseek/deepseek-v4.1-flash", ..., "reasoning_effort":"minimal"}'
HTTP 400  {"error":{"message":"Invalid option: expected one of \"low\"|\"medium\"|\"high\"|\"xhigh\"|\"max\"","type":"invalid_request_error"}}

$ jq -r '...' ~/.codex/opencodex-catalog.json
high,max,ultra          # after ocx sync, from the shipped table

Upstream documentation

https://commandcode.ai/docs/provider (OpenAI-compatible endpoint; no public per-model effort table). The model profile pages the table's profileUrl points at render client-side and ship an empty reasoningEfforts payload, which is why the in-file note about a dead self-refresh matches what I measured too.

Suggested mapping or implementation notes

Replace the affected rows (and add the 38 missing ones) with the measured ladders above; they are per-model measurements, not family inferences. If it helps, I can open a PR containing the full 46-model table plus the image-modality list, generated from the probe results.

Also relevant for the default rows: on the GOAT plan these return 403 permission_error MODEL_NOT_IN_PLAN (Pro and above, or Provider): all claude-*, gpt-5.4, gpt-5.4-mini, gpt-5.5, gpt-5.3-codex, gpt-5.6-terra, google/gemini-3.1-flash-lite, google/gemini-3.5-flash, google/gemini-3.5-flash-lite, google/gemini-3.6-flash, meta/muse-spark-1.1, sakana/fugu-ultra. /provider/v1/models lists every model regardless of plan access.

Additional context and attachments

No HAR/screenshot needed: the disagreement is between the shipped table and live HTTP responses, reproducible with the two curl commands above. Locally I carry the measured table as an idempotent patch script that re-runs after each ocx update; I would rather delete it in favour of an upstream fix.

Checks

  • I searched existing provider and compatibility issues.
  • The request and response were redacted.
  • The expected behaviour is based on an upstream specification or a concrete client requirement.

Activity

  1. added
    catalogModel catalog, slugs, visibility, routed entries
    providerProvider adapters, OpenAI-compat presets, upstream API quirks
    serviceService lifecycle (WinSW/launchd/scheduler)
    on Sep 19, 2026
  2. Ingwannu commented on Sep 20, 2026

    @Ingwannu
    Owner

    Accepted as a verified catalog-data gap. The live measurements are more specific than the shipped table and explain why pinned rows cannot be widened by config.

    Implementation should update the single Command Code effort SSOT, include the full measured model id and verification date/source, and test non-contiguous ladders (especially highspeed and medium-only cases), unknown live rows, slash/dash aliases, and synthetic ultra derivation. Do not turn this into startup probing or assume one account tier's roster/capabilities apply to every credential without documenting that scope. Keeping open for a focused data PR.

  3. HeiTuz commented on Sep 22, 2026

    @HeiTuz
    Contributor

    Rechecked the metadata side on OpenCodex 2.61.0 / Codex 0.155.1. The installed COMMAND_CODE_MODEL_EFFORTS still lists deepseek/deepseek-v4.1-flash as high,max. The generated catalog contains high,max,ultra. GLM-5.3 FlashX, Gemini 3.8 Flash and MiMo 2.6 variants currently advertise an empty ladder. This is a configuration/source check; I did not repeat the upstream request sweep, so I cannot independently confirm all accepted tiers described above.

    A diagnostic trap when reproducing:

    ocx effort model command-code/deepseek-deepseek-v4.1-flash
    # empty ladder; all sample efforts omitted/unsupported
    
    ocx effort model command-code/deepseek/deepseek-v4.1-flash
    # high,max; max -> max; ultra -> max
    

    The former is the actual Codex catalog slug; the latter includes the exact upstream ID. inspectModelEffort in src/cli/effort.ts splits at the first slash and looks up the remaining string directly, without resolving catalog slugs. The misleading first result is therefore not proof that valid requests lose their effort in the real adapter. The CLI should resolve both selectors consistently or explicitly reject unsupported selector forms rather than report an unrelated empty ladder.

  4. added
    priority: P2Medium: provider/client-specific bug with a workaround, bounded enhancement tied to a tracked issue,
    on Sep 24, 2026
  5. devin-ai-integration commented on Sep 24, 2026

    @devin-ai-integration
    Contributor

    Triage: priority: P2 — command-code effort ladders narrower than live API.

    Criteria (P2): Medium: provider/client-specific bug with a workaround, bounded enhancement tied to a tracked issue, perf, or CI reliability.

    Matched pull requests:

  6. lidge-jun commented on Sep 26, 2026

    @lidge-jun
    Owner

    Status update, keeping this open for the data half. The "config cannot widen a pinned row" part is fixed on dev: an operator ladder marked with modelReasoningEffortsAuthoritative (#5198, 98b9b34) now reaches the wire in both directions (tests/providers/command-code-provider.test.ts), and #5739 fixed ocx effort model resolving Command Code catalog slugs. The shipped table still disagrees with the 2026-09-17 measurement for deepseek-v4-flash, deepseek-v4.1-flash, both GLM-5.3 rows and gemini-3.7-flash, and several live models (MiMo, Kimi, MiniMax, grok-4.5, laguna) still have no row. That needs a focused data PR against a fresh capture.

  7. lidge-jun commented on Sep 26, 2026

    @lidge-jun
    Owner

    Fixed on dev by #5952, carried in bug-PR merge train batch 9C, #5987 (merge 81aea0f043). Command Code models now use the calibrated effort ladders measured against the service, and the first rejected rung is retried once on the filtered ladder instead of failing the turn. Closing; please reopen with the model and effort if a build that includes 81aea0f043 still rejects an advertised effort.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    catalogModel catalog, slugs, visibility, routed entriespriority: P2Medium: provider/client-specific bug with a workaround, bounded enhancement tied to a tracked issue,providerProvider adapters, OpenAI-compat presets, upstream API quirksprovider-compatibilityProvider compatibility reportsserviceService lifecycle (WinSW/launchd/scheduler)

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions