[yaoyu-33]

Stack — environment server migration, phase 1 (bottom → top): main → this PR → #4953 (configs) → #4954 (retire agents) → #5108 (resources bindings) → #5003 (DeepAgents). #4949 is merged; review this PR against main.

What and why

About 317 configs reach simple_agent through the legacy_agent relay. Under the relay the whole episode is one /run, so a crash repeats all of it. Under single_agent_turn_legacy, a crash repeats only the current step, and simple_agent resumes inside its turn. This PR adds the tooling for that switch, one small server change it needs, and a fix to four harness-variant recipes that would otherwise break. It changes no simple_agent config; #4953 applies the switch.

1. single_agent_turn binds the agent's resources server when it names none (environment_servers/single_agent_turn)

A relay names no resources server; the agent's /run uses its own. A replacement server that pins one breaks benchmarks that reach it through an _inherit_from rename of the agent, such as indic/math_500 through math_with_judge_environment_server. The rename also renames the resources server, and Gym redirects only agent references, so the pinned name dangles (ServerRefNotFoundError). resources_server is now optional. When it's unset, the server binds the resources server its agent names, as the relay did. A server where neither the server nor its agent names one fails at startup, not on the first episode. This fallback is transitional: #5108, higher in this stack, makes the Environment Server own resources_server again and has configuration resolution follow the resources rename instead.

2. scripts/add_legacy_agent_environment_servers.py --switch-simple-agent

• It replaces each relay in front of a simple_agent with a single_agent_turn_legacy server of the same name, so rows keep their routing. The server grants direct HTTP access to the resources server's tools, the routes the agent's /run called.

• Each config is resolved with its own config_paths, as Gym does, because instance names are unique only within a run.

• A child relay (_inherit_from another relay) keeps inheriting. The child's diff is just legacy_agent: → single_agent_turn_legacy:, so the parent stays a template that composition consumes. The script refuses to switch a parent without its children, or a child without its parent (exit 2).

• Before writing, it resolves every run (the repository's configs plus the given ones) as it would look after the switch. No merged instance may end up with two Environment Server types:

  • a relay that another config merges with a same-name legacy_agent server, or that a bare _inherit_from child reaches, stays on the relay;
  • if that relay inherits one that switches, it can't stay alone, so the script exits 2 and names the config, which should declare a server of its own.

• A relay stays, listed with its reason, when:

  • its agent binds no resources server;
  • its agent declares a grant simple_agent sessions reject (a required MCP access, or a second direct HTTP access). No shipped config hits these grant limitations;
  • its resources server's seed_session accepts only the legacy body, not ResourcesSeedSessionRequest; 19 shipped relays today;
  • its resources server's request models read the row's _ng_rollout_index or _ng_group_*, which an Environment Server does not forward; genrm_compare, 2 relays, whose /verify would 422;
  • its resources server's source isn't under resources_servers/, so it can't be checked;
  • the relay sets keys the switched block would drop, such as port;
  • its run cannot load an include (1 shipped relay: ruler in nemotron_3_ultra, whose include is missing on main).

  The source checks read the server's code, so a server passes once it migrates.

• Every rewrite is built before any file is written, so a refusal or an unfindable stanza (exit 2) leaves nothing half switched.

• The default (append) mode is unchanged.

3. The hle, hle_diamond, and apex_shortlist opencode/pi recipes preserve their routing names

benchmarks/{hle,hle_diamond,apex_shortlist}/{opencode,pi}.yaml (and the _search variants that include them) reused the benchmark's server by overriding it as legacy_agent in front of their harness. Once that server switches, the merge holds both types, and the server never starts. These harnesses have no agent sessions, so each recipe keeps the benchmark's original child Environment Server name and explicitly replaces its definition with a complete legacy_agent relay. _delete_key: _inherit_from detaches the override from the benchmark template, and _delete_key: single_agent_turn_legacy removes the migrated type. Only the generic parent template is set to null. The generator applies these deletions after overlay files merge and before inheritance, matching Gym. All ten recipes keep their original routing names and effective participant configuration before and after #4953.

The docs page (troubleshooting/configuration.mdx) gains a "Move a simple_agent Off the Relay" section listing these reasons. For a harness override that the script refuses, the docs and the refusal message describe this same-name override rather than a server under a new name, which would change the rows' routing.

Design

Markers: green NEW, orange CHANGED, red dashed BROKEN; the label text says the same, so the diagrams read without color.

1. Which resources server the switched server binds

Rejected design: pin resources_server on the switched server. It breaks a benchmark that reaches the server through an agent rename. Gym redirects the server's agent reference, but not its resources reference.

flowchart LR
    subgraph src["resources_servers/math_with_judge/configs/math_with_judge.yaml"]
        R0["math_with_judge<br/>(resources template)"]
        A0["math_with_judge_simple_agent"]
        E0["math_with_judge_environment_server<br/>single_agent_turn_legacy<br/>pins resources_server: math_with_judge"]
    end
    subgraph run["indic/math_500 run (config_paths includes the file above)"]
        R1["indic_math_500_..._resources_server<br/>_inherit_from: math_with_judge"]
        A1["indic_math_500_..._simple_agent<br/>_inherit_from: math_with_judge_simple_agent"]
    end
    R0 -. "consumed as template" .-> R1
    A0 -. "consumed as template" .-> A1
    E0 -- "agent_server: Gym follows the rename" --> A1
    E0 -- "resources_server: no rename following" --> X["BROKEN: math_with_judge is not in the merged run<br/>(consumed as a template)<br/>ServerRefNotFoundError"]
    class E0 changed
    class X removed
    classDef new fill:#e6f4ea,stroke:#1e8e3e,stroke-width:2px,color:#111
    classDef changed fill:#fff4e5,stroke:#e8710a,stroke-width:2px,color:#111
    classDef removed fill:#fce8e6,stroke:#d93025,stroke-width:2px,stroke-dasharray:5 5,color:#111

Loading

This PR: leave it unset, and the server binds the agent's own resources server, as the relay's /run did.

flowchart LR
    subgraph src["resources_servers/math_with_judge/configs/math_with_judge.yaml"]
        R0["math_with_judge<br/>(resources template)"]
        A0["math_with_judge_simple_agent"]
        E0["math_with_judge_environment_server<br/>single_agent_turn_legacy<br/>NEW: no resources_server"]
    end
    subgraph run["indic/math_500 run"]
        R1["indic_math_500_..._resources_server"]
        A1["indic_math_500_..._simple_agent<br/>resources_server: indic_..._resources_server"]
    end
    R0 -. "consumed as template" .-> R1
    A0 -. "consumed as template" .-> A1
    E0 -- "agent_server: Gym follows the rename" --> A1
    A1 -- "resources_server" --> R1
    E0 -- "NEW: binds the agent's own resources server<br/>(as the relay's /run did)" --> R1
    class E0 new
    classDef new fill:#e6f4ea,stroke:#1e8e3e,stroke-width:2px,color:#111
    classDef changed fill:#fff4e5,stroke:#e8710a,stroke-width:2px,color:#111
    classDef removed fill:#fce8e6,stroke:#d93025,stroke-width:2px,stroke-dasharray:5 5,color:#111

Loading

2. What --switch-simple-agent does with each instance

flowchart TD
    S["for each top-level instance in the given configs<br/>resolve its run: the file plus its config_paths, then _inherit_from"] --> Q1{"a legacy_agent relay<br/>in front of simple_agent?"}
    Q1 -- no --> L["leave it"]
    Q1 -- yes --> Q2{"blocker?"}
    Q2 -- "run cannot load an include" --> K["stays on the relay<br/>printed with its reason"]
    Q2 -- "agent binds no resources server, or declares<br/>a required MCP grant or a second direct HTTP grant" --> K
    Q2 -- "resources server: seed_session takes only the legacy body,<br/>NEW: request models read _ng_* rollout coordinates,<br/>NEW: or its source cannot be found" --> K
    Q2 -- "NEW: relay sets keys the new block would drop" --> K
    Q2 -- none --> Q3{"every parent and child<br/>switch together?"}
    Q3 -- no --> E["exit 2, names the unpaired relays"]
    Q3 -- yes --> Q4{"NEW: resolve every run after the switch:<br/>an instance with two server types?"}
    Q4 -- "yes, and the relay itself is the split" --> K
    Q4 -- "yes, through a relay it inherits" --> E2["NEW: exit 2, names the config<br/>that should declare its own server"]
    Q4 -- no --> Q5{"relay has its own _inherit_from?"}
    Q5 -- "no: parent / standalone" --> P["full block<br/>single_agent_turn_legacy, same name<br/>agent_server, direct_http grant,<br/>timeouts, no resources_server"]
    Q5 -- "yes: child" --> C["key-only rewrite<br/>keeps _inherit_from and the agent name"]
    P --> W["build every rewrite, then write in place<br/>(--check: report only, exit 1 if anything would switch)"]
    C --> W
    class P,C new
    class K,Q4,E2 changed
    classDef new fill:#e6f4ea,stroke:#1e8e3e,stroke-width:2px,color:#111
    classDef changed fill:#fff4e5,stroke:#e8710a,stroke-width:2px,color:#111
    classDef removed fill:#fce8e6,stroke:#d93025,stroke-width:2px,stroke-dasharray:5 5,color:#111

Loading

3. A harness variant of a switched benchmark

Before: the variant overrides the benchmark's server. After the switch, the merge holds both types.

flowchart LR
    subgraph base["benchmarks/hle/config.yaml"]
        E0["hle_equivalence_llm_judge_environment_server<br/>_inherit_from: equivalence_llm_judge_environment_server<br/>single_agent_turn_legacy (after #4953)"]
    end
    subgraph variant["benchmarks/hle/opencode.yaml (includes config.yaml)"]
        O0["same name<br/>legacy_agent: agent_server: hle_opencode_agent"]
    end
    E0 --> M0["BROKEN: merged instance has<br/>single_agent_turn_legacy + legacy_agent<br/>Almost-Server, not started"]
    O0 --> M0
    class M0 removed
    classDef new fill:#e6f4ea,stroke:#1e8e3e,stroke-width:2px,color:#111
    classDef changed fill:#fff4e5,stroke:#e8710a,stroke-width:2px,color:#111
    classDef removed fill:#fce8e6,stroke:#d93025,stroke-width:2px,stroke-dasharray:5 5,color:#111

Loading

This PR: the variant preserves the benchmark's routing name and replaces its inherited definition.

flowchart LR
    subgraph variant["benchmarks/hle/opencode.yaml"]
        N["CHANGED: generic parent template: null<br/>child _delete_key: _inherit_from<br/>environment_servers _delete_key: single_agent_turn_legacy"]
        O1["hle_equivalence_llm_judge_environment_server (name kept)<br/>legacy_agent, entrypoint app.py"]
        A["hle_opencode_agent<br/>opencode_sandboxed_agent (no agent sessions)"]
    end
    O1 -- "agent_server" --> A
    class N changed
    class O1 new
    classDef new fill:#e6f4ea,stroke:#1e8e3e,stroke-width:2px,color:#111
    classDef changed fill:#fff4e5,stroke:#e8710a,stroke-width:2px,color:#111
    classDef removed fill:#fce8e6,stroke:#d93025,stroke-width:2px,stroke-dasharray:5 5,color:#111

Loading

The runtime call flow of a switched episode (relay vs single_agent_turn_legacy) is drawn in #4953, where the shipped configs change.

Validation

• tests/unit_tests/test_add_legacy_agent_environment_servers.py: 99 passed. The new cases cover:

  • replacing a relay, then parsing it strictly and validating it against SingleAgentTurnEnvironmentServerConfig;
  • --check;
  • relays in front of other agents left alone;
  • stay-on-relay reasons: MCP, a second direct HTTP grant, unbound, a missing include, legacy, typed, raw-typed and base seed_session, rollout coordinates, a missing implementation, and settings the new block would drop;
  • a child keeping its parent, and refusal for an unpaired child or parent;
  • a same-name legacy_agent override (stays), the same relay declared in two files and then overridden (both declarations stay), the same override of an inheriting relay (exit 2), and a variant with its own relay (the benchmark switches);
  • a bare _inherit_from child of a relay that stays;
  • an unfindable stanza writing nothing;
  • a switched relay following an agent rename;
  • the ten harness recipes keeping their routing names and participant configs across the switch. Without the hle_diamond overrides, its four cases fail with AlmostServerError.

  The rename test reproduces the indic/math_500 ServerRefNotFoundError against a version that pins resources_server.

• environment_servers/single_agent_turn/tests: 119 passed, including the fallback and the startup failure. single_agent_turn_legacy/tests: 23 passed, including the task_source default, the bound check and aggregate_metrics forwarding with resources_server unset.

• Repo-wide --switch-simple-agent --check: 295 would switch and 22 stay (19 seed, 2 genrm_compare, 1 missing include), exit 1, no refusals. The default --check adds nothing.

• Gym's config parser on the ten hle / hle_diamond / apex_shortlist opencode/pi recipes, on this PR and after chore(configs): move simple_agent configs off the legacy_agent relay #4953's switch: one legacy_agent server each, fronting the harness. For hle_diamond, the resolved participants of its four recipes, and of opencode and pi_search with judge_openai.yaml and opencode with sandbox_docker_local.yaml, are byte-identical to the head before its override.

• pre-commit run --files <changed files>: clean.

Preserved-name validation

• The complete generator test file passes all 95 tests, including same-name harness migration and Gym-compatible deletion ordering.
• pre-commit run --all-files, git diff --check, and DCO sign-off checks passed.
• The six HLE/APEX Pi, OpenCode, and HLE search entrypoints were compared with the pre-feat(scripts): switch simple_agent relays to single_agent_turn_legacy #4952 baseline at all four updated stack layers: 30 actual-parser resolutions preserve the original Environment Server names and all effective server/sandbox settings. No new real-model rollout was run for this restack; existing migration and model-qualification gaps remain.
• The combined stack at fix(env): own resources bindings and resolve inherited renames #5108 also passes 1,184 focused config, generator, environment, agent, checkpoint, and rollout tests. This supplements, rather than replaces, the per-layer checks.

Rollout evidence

N/A for a direct rollout here: the switch reaches episodes through #4953's configs, where the smoke runs are recorded. A real-model smoke of the earlier heads reproduced the two failures this PR's checks now catch: genrm_compare verifiers returning 422 after successful policy calls, and the six variant recipes no longer resolving. An independent re-check of the fixed heads found both resolved, plus one generator edge (the same relay declared in two files, then overridden, raised StopIteration without writing), now fixed and tested.

Compatibility

• single_agent_turn configs that set resources_server behave as before. Leaving it unset was a validation error and now binds the agent's resources server. A config that inherits a shipped single_agent_turn* template, which sets resources_server.name: ???, sets resources_server: null to get this.
• The ten opencode/pi variant runs preserve hle_equivalence_llm_judge_environment_server, hle_diamond_equivalence_llm_judge_environment_server, or apex_shortlist_math_with_judge_environment_server. Existing references keep working; no instance rename is required.
• Not changed: resolve_dataset_agent still requires an explicit resources_server for a resources-declared taskset, as under the relay.

Update (2026-10-28)

• Rebased onto main after feat(environment): honor skip_verification in single_agent_turn #4949 merged.
• hle_diamond (feat(eval): add the HLE-Diamond benchmark with text and sandboxed search presets #4727, merged 2026-10-24) uses the same harness-override pattern in opencode.yaml and pi.yaml, which its _search variants include. On the rebased head, the repo-wide --switch-simple-agent --check refused all four (exit 2). They now get the same-name override, and the recipe test covers them.
• The docs paragraph and the refusal message for that case now describe the same-name override.
• Validation above, on 84e0a93ef with Python 3.13.14: 241 passed across the generator, single_agent_turn, and single_agent_turn_legacy tests. pre-commit run --files passed on the hle_diamond change, and ruff check and ruff format --check pass on the updated script.
• CI on ad8912056: the failures were unrelated to this change. They were Docker Hub pull rate limits (harbor_agent_general in server shard 7, and test_harbor_image_build), a git ls-remote timeout (test_orchestration_otel), and the non-blocking kilocode conformance probes, which fail the same way on other open PRs and in the merge queue.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yaoyu-33]

/ok to test 2a31fc2

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-4952-84e0a93ef875.docs.buildwithfern.com/nemo/gym

[yaoyu-33]

Team review of the generator and the single_agent_turn fallback. The fallback itself checks out: every former config.resources_server.name use (seed, grant, verify, close, aggregate_metrics, the legacy task_source default and bound check) now goes through resources_server_name, checkpoint resume re-derives the same name, and a repo-wide --switch-simple-agent run is byte-identical to #4953 and idempotent.

The main gap is in the generator's safety checks: it approves switches that leave a merged instance with two Environment Server types, and it switches servers whose /verify needs row coordinates the Environment Server does not forward. Both show up as breakage in #4953.

Not inline (outside the diff):

• nemo_gym/global_config.py:2172-2182 (resolve_dataset_agent): a resources-declared taskset still requires an explicit resources_server on the routed Environment Server, so the new fallback is not honored there. It is not a regression (the relay had none either), but the README now says unset is fine. A fallback or a sentence would close it.
• The base branch from #4949 has moved and now touches single_agent_turn/app.py, so this needs a rebase.
• The validation list would be stronger with a resolve pass over every config whose run includes a switched file (that is how the issue in the first comment shows up).

[yaoyu-33]

/ok to test b86400f

[yaoyu-33]

/ok to test b86400f

[copy-pr-bot]

/ok to test b86400f

@yaoyu-33, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

[yaoyu-33]

/ok to test 52e2e2c

[weijiac0619]

/ok to test ad89120

[weijiac0619]

/ok to test 84e0a93

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/40780076433

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/40785007489