feat(ai): validate catalog generation invariants and fail violating regens

[snimu]

Purpose

Make catalog regeneration self-checking: a regen that violates known invariants fails loudly and refuses to write, instead of shipping bad rows that surface as user-facing 400s, early compaction, or dead /effort levels. Fixes the current violations by regen-time derivation, not hand-edits. Squashes #1683/#1680, #1406, #1597, the live case of #1725, and guards the #1679/#1724 staleness class at the generator.

Mechanism (new scripts/validate-model-catalog.ts, wired into generate-models)

Four invariants, checked on the deduped catalog before it is written; violations print by name and the generator exits nonzero without writing:

1. maxTokens <= contextWindow on every row ([Bug] Generated model catalog contains impossible maxTokens > contextWindow values #1683/[Bug] Generated catalog contains models where maxTokens exceeds contextWindow (Inkling-Small, openrouter gpt-3.5-turbo-0613) #1680).
2. Every github-copilot id must match an explicitly classified family, and the row's api must equal that classification. Unclassified families fail by name instead of silently defaulting to chat completions — the exact mechanism that shipped the Grok misroute (GitHub Copilot grok-4.5 is routed to Chat Completions during model generation #1406).
3. openai-codex rows may not diverge more than 2x from the same model's openai contextWindow ([Bug] openai-codex applies one flat 272k context window to every model, including 1M-context models #1597/[Bug] Cross-provider catalog limit contradictions cause silent truncation and premature compaction #1724 class).
4. Same-model rows across providers must agree on selectable thinking levels ([Bug] thinkingLevelMap inconsistently populated across providers for the same model - /effort diverges by provider #1725). The off level is excluded from comparison: whether thinking can be disabled legitimately varies per transport (reasoning effort none exists only on native OpenAI Responses; the generator encodes that deliberately per provider).

generateModels().catch now sets a nonzero exit code (previously catch(console.error) exited 0 even on total failure).

Data fixes (all by derivation; catalog regenerated on the rebased sources)

fix	derivation	rows vs main
Copilot Grok + MAI-Code 400 on /chat/completions (#1406)	copilotModelApi routes grok-/mai- to openai-responses (upstream pi-mono #906)	4
Codex GPT-5.6 family stuck at the flat 272k (#1597)	codex contextWindow derives from the openai row when the API side serves 1M+. The 1,050,000 lift for the 5.6 rows rests on the #1597 backend measurement (the ChatGPT backend accepting >272k for 5.6); upstream Codex CLI models.json (rust-v0.153.4) still lists them at 272000, so the citation justifies only the gpt-6-astra exception (CODEX_SMALLER_WINDOW_VERIFIED, kept at 272000 and excluded from the >2x invariant)	3 → 1,050,000
thinkingmachines/Inkling-Small maxTokens > contextWindow (#1683)	models.dev ingestion clamps output limits to the context window	1
Kimi K3 thinking levels (#1725)	shared map corrected to the low/high/max the model supports; per-transport outcome: 3 effort-capable completions rows (huggingface, opencode, opencode-go) gain low/high/max; 3 effort-incapable rows (Copilot, Moonshot, Moonshot CN) go all-null; 6 anthropic-messages rows (Fireworks ×2, Kimi Coding ×2, Vercel ×2) gain low/high with max budget-clamp-nulled. openrouter/prime already carried the rich map — unchanged	12
Budget-clamped phantom levels on non-adaptive anthropic-messages rows (review round)	non-adaptive anthropic rows serialize thinking as budget tokens where clampReasoning folds xhigh/max into high; generation nulls those entries and a new invariant rejects the class. Covers the 6 Kimi anthropic rows above plus 23 pre-existing Vercel openai-via-anthropic rows (the RES-1276 routing oddity)	29 (23 beyond the Kimi rows)

Intended-fix total: 43 rows vs main (4 + 3 + 1 + 12 + 23). Validator sanity vs the pre-fix committed catalog (post-#2069/#2074 main): reports exactly the known defect set and nothing else; zero violations on the regenerated output. Regen byte-stable across a double run. The rebased regen also carries live upstream drift (~110 rows beyond the fixes): a Prime Inference listing refresh (display-name normalization across ~70 rows, 10 additions, gpt-5.6-luna route removals, context/maxTokens updates), models.dev dropping the six legacy moonshot kimi-k2 preview rows per provider (verified upstream: only k2.6/k2.7 remain — real removals, not a fetch failure), Bedrock adding grok-4.6, and OpenRouter price/listing churn. This is regen-reflects-fetch-time data, the same class the deterministic-build PR (#2038) exists to make explicit.

Tests

• New test/model-catalog-validation.test.ts (6 tests): one pin that the committed catalog passes with zero violations (this is the standing regression guard for all four invariants), one fixture-violation test per invariant proving it fails loudly, and one pin that off-level variance alone is accepted.
• supports-xhigh.test.ts: existing 272000 pin updated to 1050000 (fail-unfixed for the codex fix).
• E2E fail path exercised in a sandbox: reintroducing the kimi drift makes npm run generate-models exit 1 with the named violation and leaves the committed catalog untouched.

Premise corrections vs the RES-1273 plan

• validate-model-catalog.ts did not exist on main; feat: refresh Prime Inference models from live API #1928 introduces a 9-line schema wrapper at the same path on its branch. This PR creates the invariant validator there with composable exports (validateModelCatalog(catalogLike)), so feat: refresh Prime Inference models from live API #1928 can call the same invariants on its hosted aggregate after rebase. Heads-up left for coordination: same path, trivial to reconcile, no second validator.
• defaults-exist is deliberately NOT a generator invariant: defaultModelPerProvider lives in packages/coding-agent, and importing it from the ai generation script requires a built pi-ai dist (generation must work pre-build) — a fragile reverse dependency. Open fix(ai): omit the default service tier, reprice cache writes from message_delta, repoint the zai default #2032 already adds the catalog-existence pin at the right layer (coding-agent tests), which fails in the same PR as any violating regen. Duplicating it in the generator would be a wrong-layer second checkpoint bought with a brittle import.
• The plan counted 2 current violations; the requested invariants actually surface 3 in-scope (Inkling-Small, codex 5.6 contexts, kimi-k3 thinking levels — all fixed here) plus the pre-existing zai default gap owned by fix(ai): omit the default service tier, reprice cache writes from message_delta, repoint the zai default #2032.
• A full cross-provider thinkingLevelMap merge pass (propagating maps to the ~70 families where blind sources emit none) is deliberately out of scope: it is a large behavioral change requiring per-transport verification, and only one family conflicts today. The invariant now makes any future conflict loud.

Size

Total src: +175/−16 (net +159); tests: +156/−1 (net +155).

Linear: RES-1273

---

Note

Medium Risk
Changes model API routing and catalog metadata used at runtime; bad invariants now block releases, but incorrect classification would break Copilot or effort UI for affected models.

Overview
Adds catalog generation guards so generate-models runs validateModelCatalog before writing models.generated.ts and exits with a failure (nonzero exit code) when invariants break, instead of shipping bad rows.

The validator and generator now derive fixes for known bad metadata: clamp maxTokens to contextWindow, classify GitHub Copilot models via shared copilotModelApi (including Grok/MAI → openai-responses), lift openai-codex GPT-5.6 family context to match 1M+ openai twins (except verified-smaller gpt-6-astra), align Kimi K3 thinking maps per transport (low/high where effort is sendable; all-null where it is not), and strip phantom xhigh/max levels on budget-token Anthropic paths and on completions transports that cannot send reasoning effort.

Coding-agent behavior is adjusted so switching to a reasoning model with no selectable thinking levels no longer persists off as the user default, and cycling effort on such a model is a no-op. Tests pin the committed catalog as valid and cover each invariant plus the session thinking-default fix.

^{Reviewed by Cursor Bugbot for commit e4e6cff. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add catalog invariant validation to generateModels and fix thinking-level bugs

• Adds validateModelCatalog in validate-model-catalog.ts which checks token-window, Copilot-routing, Codex-window, transport-capability, Anthropic-budget, and cross-provider thinking-level invariants; generation now throws and exits with status 1 on any violation
• Fixes Copilot classification so Grok and MAI families route to OpenAI Responses via copilotModelApi
• Syncs Codex context windows with matching OpenAI rows when the OpenAI window is ≥1M tokens, with verified exceptions (e.g. GPT-6 Astra) in CODEX_SMALLER_WINDOW_VERIFIED
• Fixes AgentSession.setThinkingLevel, cycleThinkingLevel, and _getThinkingLevelForModelSwitch so reasoning models with no selectable levels no longer overwrite the persisted thinking default with off
• Risk: catalog generation now fails the build on invariant violations; any future model additions must satisfy all checks in validateModelCatalog or generation will exit nonzero

^{Macroscope summarized e4e6cff.}

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit b7362bf. Configure here.}