Repository navigation
feat(openrouter): add catalogue-wide model support - #586
Conversation
2810161 to
308a57f
Compare
Add a first-class OpenRouter agent across configuration, runtime, Modal, evaluation workflows, schemas, tests, and documentation. Preserve arbitrary OpenRouter catalogue model identifiers and isolate OPENROUTER_API_KEY credentials through the Codex-backed adapter. Closes #584.
308a57f to
4d531e6
Compare
v0.1.0 integration policy — 2026-08-14OpenRouter is an explicitly approved exception to the v0.1.0 feature freeze. This PR remains targeted to and must merge directly into This is a scope clarification, not merge authorization. Current blockers include the conflict with Merging this PR would not itself authorize a live paid OpenRouter run, the v0.1.0 benchmark, tagging, or publishing. |
|
Note: we're waiting on OpenRouter admin to remove sensitive filters, otherwise we can't test this |
Every one-shot validator worker eagerly compiled all 65 registered schemas before validating a single root. Ajv compilation of the full bundle costs ~540ms against ~39ms for one root, so each `validateJsonFile` call paid a ~14x setup penalty. On the four-core CI runner that pushed four Modal package-gate tests past their existing timeouts. The eager pass was added to keep referenced-schema diagnostics stable, but main fixed the same diagnostics drift in #595 by correcting the expected schema paths, so the compile-all is now redundant. Ajv already compiles a root together with its reachable references on first `getSchema`, and the host and one-shot workers project identical paths without it. Modal package gate on four pinned cores: 561 passed in 97s, versus four timeouts in 364s before. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
OpenRouter rate-limits the routed Codex model hard enough that the smoke lane's six concurrent CodexAgent calls exhaust Codex's HTTP retries with 429 responses before any target starts. Bound an OpenRouter runner to one eval row and one workflow node; every other provider keeps the lane's checked-in concurrency. The bound is a single checked-in constant so that all four trusted policy derivations agree: the Modal worker's suite, the producer manifest, the launch guardrail, and the cleanup and publication re-derivations. Cleanup now takes a policy-dimension resolver so it reads the dispatched provider off the manifest exactly as the launch guardrail does, instead of failing validation and leaving Modal sandboxes running. Also drop the temporary OpenRouter diagnostic probe job and scripts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Summary
OpenRouterAgentusingOPENROUTER_API_KEYandhttps://openrouter.ai/api/v1Validation
pnpm -w format:checkpnpm --filter @ultrafuzz/config test(62 tests)pnpm --filter @ultrafuzz/runtime typecheckCloses #584.
Greptile Summary
The PR adds first-class OpenRouter support across configuration, generated runtime adapters, local and Modal execution, benchmark provenance, schemas, workflows, and documentation.
Confidence Score: 4/5
The PR is not yet safe to merge because Doctor can still approve credential posture for a configured topology while a run-time OpenRouter topology override immediately fails preflight.
Doctor has no input for the run’s
--topology-pathand derives credential requirements from the configured topology, whereas start-run validates credentials against the override-expanded graph; the existing topology-override diagnostic mismatch therefore remains.Files Needing Attention: packages/runtime/src/doctor.ts, packages/runtime/src/types.ts
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR Config[OpenRouter model profile] --> Validate[Config and topology validation] Validate --> Graph[Expanded run graph] Graph --> Preflight[Credential and command preflight] Preflight --> Agent[Generated OpenRouterAgent] Agent --> Codex[Codex CLI] Codex --> Router[OpenRouter API] Graph --> Modal[Modal benchmark execution] Modal --> Evidence[Benchmark and publication evidence]Reviews (12): Last reviewed commit: "fix(benchmarks): serialize the OpenRoute..." | Re-trigger Greptile
Context used: