You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Teams using Switchyard (or similar weak/strong routing) need to systematically measure how routing affects benchmark scores, failures, or cost. Gym can proxy to Switchyard today, but routing policy isn't a first-class eval variable: no comparable runs, no attribution, no guided workflow.
Gym already does this for other knobs (e.g. skills). Routing should work the same way.
Note: These changes may have cross-repo impact: Switchyard + Gym
User outcomes
Run the same benchmark under different routing conditions and compare scores + token/cost signals fairly.
Identify whether routing helps or hurts on tasks they care about
Reproduce routing experiments like any other Gym eval.
Scope Notes
Enable routing conditions as experimental variables in Gym evals, with outputs that support comparison and diagnosis.
Activity
docs: make the Switchyard page discoverable and its deployment writable
docs: make the Switchyard page discoverable and its deployment writable
feat: record the routing condition an eval runs under
docs: make the Switchyard page discoverable and its deployment writable
feat: record the routing condition an eval runs under
feat: add switchyard_model server for router-backed evals with condit…
phtran8 commentedon Aug 26, 2026
Hi @cwing-nvidia , please help share SWITCHYARD_API_KEY for QA testing.