[yaoyu-33]

What and why

A --agent-type harness swap (harness ablation, fern/versions/latest/pages/evaluation/harness.mdx) onto an environment fronted by single_agent_turn_legacy kept that server type even when the incoming agent does not implement agent sessions (opencode_sandboxed_agent, pi_sandboxed_agent, the swe agents, non_executing_simple_agent, tool_simulation_agent, ...). The config resolved cleanly, then every episode failed with NotImplementedError from /v1/agent_sessions. Behind a legacy_agent relay, the same swap worked through the agent's /run.

This blocks #4953, which moves ~285 simple_agent configs to single_agent_turn_legacy. It is independent of the rest of that stack, so it is based on main.

Changes

• compose_unbound_agent (in nemo_gym/global_config.py): when the swapped-in agent type has no session support, each single_agent_turn_legacy server fronting a composed agent is rewritten to a legacy_agent relay. The relay keeps entrypoint, agent_server (already retargeted), and host/port/num_workers. It drops the session-only keys (resources_server, resources_tool_transports, agent_outcome_policy, the episode limits); legacy_agent defaults the limits, and warns if any are set. The server keeps its environment name. An info log records the fallback.
• Why fall back instead of failing: the documented ablation workflow worked for these harnesses before the migration. A relay restores exactly that behavior (the agent runs its own /run against its carried-over Resources binding). An AgentCompositionError would turn a migration that should not change behavior into a regression for every session-less harness.
• Source of truth for session support: AGENT_SESSION_AGENT_TYPES in nemo_gym/global_config.py. Composition cannot import agent apps, so this is a static list. scripts/add_legacy_agent_environment_servers.py now imports it instead of keeping its own hard-coded set. A unit test asserts the list equals the set of responses_api_agents/*/app.py that define _seed_agent_session_state, so it cannot silently drift.
  • The script's set grows from 10 to 13 types (simple_agent, nooa_agent, stirrup_agent), and only affects instances with an explicit resources_server: null. python scripts/add_legacy_agent_environment_servers.py --check reports 0 files touched, so no repo config changes.

No issue filed: found in review of the #4952 → #4953 → #4954 stack.

Validation

• pytest tests/unit_tests/test_global_config.py tests/unit_tests/test_add_legacy_agent_environment_servers.py: 304 passed. New tests:
  • session-less swap (opencode_sandboxed_agent, non_executing_simple_agent) → legacy_agent relay with only entrypoint + retargeted agent_server
  • session-capable swap (hermes_agent, opencode_agent) → stays single_agent_turn_legacy, every key preserved, agent_server retargeted
  • AGENT_SESSION_AGENT_TYPES matches the agent apps that implement _seed_agent_session_state
• Full pytest tests/unit_tests: 38 failures + 2 collection errors locally (opensandbox, reward_profile, super35 launcher, telemetry imports). The same tests fail identically on the base commit, so they are not from this change.
• python scripts/add_legacy_agent_environment_servers.py --check: 0 files touched.
• pre-commit run --files on the changed files: passed.
• Repro with chore(configs): move simple_agent configs off the legacy_agent relay #4953's benchmarks/gpqa/config.yaml and resources_servers/mcqa/configs/mcqa.yaml applied locally (not committed), gym env resolve with an openai_model config:
  • --config responses_api_agents/opencode_sandboxed_agent/configs/opencode_sandboxed_agent.yaml → gpqa_mcqa_environment_server is legacy_agent fronting gpqa_mcqa_opencode_sandboxed_agent (before: single_agent_turn_legacy)
  • opencode_agent with its resources binding left unset → stays single_agent_turn_legacy fronting gpqa_mcqa_opencode_agent

Rollout evidence

N/A: composition-only change. The resolved config is the same legacy_agent relay these swaps used on main before #4953. No model rollouts were run.

Compatibility

No change on main today: no shipped config uses single_agent_turn_legacy yet. After #4953, session-less harness swaps work again; session-capable swaps are unchanged.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.