[yaoyu-33]

Stack — environment server migration, phase 1 (bottom → top): main (with #4952 and #4953 merged) → this PR → #5108 (resources bindings) → #5003 (DeepAgents).

What and why

tool_simulation_agent and non_executing_simple_agent make one model call and pass its tool calls to the verifier unexecuted. simple_agent with execute_tools: false does the same, as an agent session. Moving their configs onto it puts them under single_agent_turn_legacy. A crash then repeats only the current step, and the agent resumes inside its turn for free.

14 configs move (13 tool_simulation_agent, 1 non_executing_simple_agent): image_tools_pivot, swe_pivot, structured_outputs_v4, seven single_step_tool_use_with_argument_comparison configs (including bc_v4_8), and four terminal_multi_harness configs. In each:

• the agent becomes simple_agent with execute_tools: false, keeping its instance name, so training configs and row agent_refs still resolve;
• the relay becomes single_agent_turn_legacy, generated with feat(scripts): switch simple_agent relays to single_agent_turn_legacy #4952's script. It has to: simple_agent refuses /run when execute_tools is false. Its timeouts come from the server's defaults (21600 s episode, 180 s cleanup), as in chore(configs): move simple_agent configs off the legacy_agent relay #4953;
• the server grants the agent no tool access (no resources_tool_transports), since a non-executing agent calls no tool. The pivot skill's template does the same;
• allowed_agents gains simple_agent. Rows routed straight to the agent are checked against it, and tool_simulation_agent stays listed while it exists.

The pivot-dataset skill's config template and its reference config writer now emit simple_agent (execute_tools: false) with a single_agent_turn_legacy server. The image_tools README pairing note is updated.

Not deleted: both agents stay, and their READMEs now say they're deprecated. Deleting them would break user configs that name these agent types, so removal is left for a later release.

Out of scope: agents not present in this public repository.

Design

Markers: green NEW, orange CHANGED, red dashed BROKEN; the label text says the same, so the diagrams read without color.

1. One episode, before and after

Before: tool_simulation_agent behind the relay.

sequenceDiagram
    autonumber
    participant RC as Rollout collection
    participant LA as legacy_agent (relay)
    participant TS as tool_simulation_agent /run (REMOVED from configs)
    participant M as Model
    participant RS as Resources server
    RC->>LA: /run
    LA->>TS: /run (whole episode)
    opt only when checkpointing
        TS->>RS: /seed_session
    end
    TS->>TS: /v1/responses (self-dispatch)
    TS->>M: /v1/responses (one call)
    Note over TS: tool calls are never executed
    TS->>RS: /verify (response with the function_call items)
    TS-->>LA: result
    LA-->>RC: result

Loading

After: simple_agent with execute_tools: false under single_agent_turn_legacy.

sequenceDiagram
    autonumber
    participant RC as Rollout collection
    participant ES as single_agent_turn_legacy (NEW for these configs)
    participant AG as simple_agent, execute_tools=false (NEW)
    participant M as Model
    participant RS as Resources server
    RC->>ES: /run (same row, same server and agent names)
    ES->>RS: /seed_session (always)
    ES->>AG: /v1/agent_sessions (no tool grant)
    ES->>AG: /v1/responses
    AG->>M: /v1/responses (one call)
    Note over AG: execute_tools=false: stop after this call,<br/>return the function_call items unexecuted
    AG-->>ES: response
    ES->>AG: /v1/agent_sessions/close
    ES->>RS: /verify (same response shape)
    ES->>RS: /close_session (always)
    ES-->>RC: flat result row

Loading

2. How each of the 14 configs changes (swe_pivot shown)

Before

flowchart LR
    E0["swe_pivot_environment_server<br/>legacy_agent"] --> A0["swe_pivot_tool_simulation_agent<br/>responses_api_agents.tool_simulation_agent"]
    A0 -- "resources_server" --> R0["swe_pivot resources server<br/>allowed_agents: [tool_simulation_agent]"]

Loading

After

flowchart LR
    E1["swe_pivot_environment_server (name kept)<br/>CHANGED: single_agent_turn_legacy<br/>(simple_agent refuses /run without tool execution)<br/>NEW: no resources_tool_transports"] --> A1["swe_pivot_tool_simulation_agent (name kept)<br/>CHANGED: responses_api_agents.simple_agent<br/>NEW: execute_tools: false"]
    A1 -- "resources_server" --> R1["swe_pivot resources server<br/>CHANGED: allowed_agents: [simple_agent, tool_simulation_agent]"]
    E1 -. "binds the agent's resources server" .-> R1
    D["tool_simulation_agent and non_executing_simple_agent<br/>still exist, READMEs marked deprecated"]
    class E1,A1,R1,D changed
    classDef new fill:#e6f4ea,stroke:#1e8e3e,stroke-width:2px,color:#111
    classDef changed fill:#fff4e5,stroke:#e8710a,stroke-width:2px,color:#111
    classDef removed fill:#fce8e6,stroke:#d93025,stroke-width:2px,stroke-dasharray:5 5,color:#111

Loading

Validation

• Every config resolves as before: gym env resolve on all 638 server configs, before (chore(configs): move simple_agent configs off the legacy_agent relay #4953) and after. The same 128 fail on both sides, and none changed status. In the structural check, 14 more relays become single_agent_turn_legacy, and all bind their agent's resources server. 0 problems. At the current head, swe_pivot, bc_v4_8 and structured_outputs_v4 resolve with single_agent_turn_legacy and no tool grant.
• A repository test (test_repository_non_executing_simple_agents_run_in_sessions_without_tool_access) asserts that no execute_tools: false simple_agent sits behind legacy_agent or gets a tool grant. It fails with the grants restored. Generator tests: 86 passed.
• Server tests:
  • single_step_tool_use_with_argument_comparison: 101 passed; bc_v4_8 wiring asserts execute_tools: false and the new server.
  • swe_pivot: 190 passed; image_tools: 32 passed.
  • terminal_multi_harness and structured_outputs don't collect in this venv (openapi_schema_validator, xmltodict are missing), the same as on the base. They read no configs, and this PR changes none of their code. On a workstation with their dependencies, they passed in isolated processes (54 and 6 tests).
• pre-commit run --files <changed files>: clean.

Preserved-name restack validation

• Rebased onto the latest chore(configs): move simple_agent configs off the legacy_agent relay #4953, including its session-less composition fallback and session tool-observation fixes. git range-diff confirms this PR's two retirement commits retain their patches.
• The six HLE/APEX Pi, OpenCode, and HLE search entrypoints were compared with the pre-feat(scripts): switch simple_agent relays to single_agent_turn_legacy #4952 baseline at all four updated stack layers: 30 actual-parser resolutions preserve the original Environment Server names and all effective server/sandbox settings. No new real-model rollout was run for this restack; existing migration and model-qualification gaps remain.
• Intermediate-branch harness/deletion and non-executing-agent wiring tests, pre-commit run --all-files, DCO trailers, and whitespace checks pass. The combined fix(env): own resources bindings and resolve inherited renames #5108 stack passes 1,184 focused tests.

Rollout evidence: relay vs session parity

These runs use a stub Responses model, not a real one: it calls the first offered tool once, then answers. The same four example rows per config went through the old path (this PR's base: the old agent behind the relay) and the new one, on the same server code.

Config	Relay rewards by task	Session rewards by task	Match
swe_pivot	0, 0, 0, 0	0, 0, 0, 0	yes
single_step_tool_use_with_argument_comparison	0, 1, 0, 1	0, 1, 0, 1	yes
structured_outputs_v4	0, 0, 0, 0	0, 0, 0, 0	yes

On the session path every row seeds, opens the agent session, makes one model call (no tool route is called), closes the agent, verifies, and closes the resources session. On the relay, the agent's /run did the episode itself.

Real model, paired. All 14 changed configs plus one public SWE crosscheck ran against #4953 (old agent behind the relay) and this PR (128288a3b, before the tool grant was dropped). Each used real data, including the original embedded images with a vision model:

• 69 task rows per side, exactly one policy request per row, and no resource-tool calls;
• identical per-row rewards (sum 25 per side), with agent and resources sessions closed on the session side.

Dropping the unused grant after that run changes no call the episode makes; an independent re-check after the change passed 14/14 paired protocol checks.

Not done: a full reward-profile comparison on a pivot training distribution. The terminal_multi_harness rows exercised the shipped Codex examples, not each harness's private distribution.

Compatibility

• Instance and server names don't change.
• Differences from tool_simulation_agent:
  • The resources server is now always seeded and closed.
  • simple_agent marks a reasoning-only reply incomplete; it is still verified.
  • A failed model call becomes a typed failure row (retryable per the outcome policy) instead of a /run error.
• skip_verification keeps working through feat(environment): honor skip_verification in single_agent_turn #4949.
• --agent-type swaps on these configs: simple_agent is now allowed. Composition doesn't carry execute_tools: false over, but without a tool grant such an agent fails loudly when it calls a tool, instead of being mis-scored. The session-less composition fallback from fix(config): relay session-less agents swapped onto single_agent_turn_legacy #4995, carried by chore(configs): move simple_agent configs off the legacy_agent relay #4953 and inherited here, routes a swapped-in tool_simulation_agent through legacy_agent.

CI refresh (2026-10-23)

Rebased onto main at 8041d1910 and preserved the existing stack bases. This includes #4973's atomic resubmit-lock fix and #4966's Terminus PROMPT_COMMAND fix, which address the failures in the previous CI runs. No additional runtime behavior changes were introduced by this refresh.

Restacked on the rebased #4953. The repository non-executing-agent wiring check and test_bc_v4_8_config.py: 14 passed. The single_agent_turn, single_agent_turn_legacy, and simple_agent app tests: 220 passed. The original migration patches are unchanged.

pre-commit run --all-files, git diff --check, and DCO trailer checks passed. Fresh Linux CI is still required on the new head. Rollout evidence for this refresh: N/A; this only rebases the existing changes onto the upstream CI fixes.

Update after #4953 merged (2026-10-30)

When #4953 merged, this PR's two commits were replayed onto main (7f9972dd0). The replay kept their authorship and messages, and git patch-id shows the same patch as before. Two commits are added on top:

• chore(configs): rely on single_agent_turn_legacy timeout defaults: chore(configs): move simple_agent configs off the legacy_agent relay #4953 moved default_episode_timeout_seconds: 21600 and cleanup_timeout_seconds: 180 into SingleAgentTurnLegacyEnvironmentServerConfig and dropped them from the migrated configs. The 14 configs here, the pivot skill example, and its reference config writer still repeated them. The effective settings are unchanged.
• chore(image_tools): drop the comment execute_tools now states.

Tests (Linux, server tests run from each server directory as CI does):

Scope	Result
Generator, test_global_config, test_benchmarks	413 passed
single_step_tool_use_with_argument_comparison	101 passed
swe_pivot	190 passed
image_tools	32 passed
terminal_multi_harness	54 passed
structured_outputs	6 passed
simple_agent	89 passed

Old vs. new rollouts. Each config was started from main (7f9972dd0: the old agent behind legacy_agent) and from this head (simple_agent with execute_tools: false under single_agent_turn_legacy). The example rows were posted to the environment server's /run against a deterministic stub model in four modes: replay of the recorded outputs, a call to the first offered tool, a message only, and reasoning only.

Config	Rows × modes	Identical reward and output items
single_step_tool_use_with_argument_comparison	5 × 4	20/20
bc_v4_8	5 × 4	20/20
swe_pivot	5 × 4	20/20
image_tools_pivot	5 × 4	20/20
terminal_multi_harness_agent006	5 × 4	20/20
structured_outputs_v4	5 × 4	20/20

No errors on either side. The only visible difference is the one listed under Compatibility: in reasoning-only mode, the new path marks the response incomplete (incomplete_reasoning), with the same reward.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yaoyu-33]

/ok to test 128288a

[yaoyu-33]

Team review of the agent consolidation. All 14 configs convert cleanly: each resolves before and after with only the intended differences (simple_agent with execute_tools: false, single_agent_turn_legacy, allowed_agents += simple_agent). Nothing is dropped (max_steps, datasets, model and resources refs). The five verifiers read only task fields, responses_create_params.tools, and response.output/error, so the one-call output reaches them unchanged.

Not inline:

• Keeping tool_simulation_agent in allowed_agents is reasonable while it is deprecated. Swapping it in with --agent-type, though, now fails on /v1/agent_sessions (it has no session support), which is the composition gap raised on #4953. That fix covers this case too.
• Only bc_v4_8 asserts the new wiring. A cheap parametrized test over the 14 YAMLs (simple_agent, execute_tools false, single_agent_turn_legacy) would guard the rest.
• Stub-model parity covers 3 of 14 configs; the real-model profile is pending, as the body says.

[yaoyu-33]

/ok to test 3e6c00b

[yaoyu-33]

/ok to test 521205f

[weijiac0619]

/ok to test 0746359

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41504443634