[ananthsub]

What does this PR do?

Runs Hermes in a Resources-owned sandbox when SandboxAccess is supplied, or creates an Agent-owned fallback sandbox otherwise. Both paths return sandbox-captured model and tool observations during close.

Hermes calls the Model Server directly from the sandbox, so the sandbox must be able to reach it. The runner points every model client Hermes builds at the rollout's Model Server URL: the root agent, its iteration-limit summary, delegated children, and auxiliary clients. Streaming is turned off because the Model Server answers only whole responses.

The sandbox installs the Hermes version pinned in requirements.txt, derived at startup from the Hermes installed with the server, so there is one version pin and the sandbox runs the same Hermes as the host. Seeding reuses that Hermes when it already imports in the sandbox, for example because the image bakes it in, and otherwise installs it and checks the import again. The runner starts with one blocking exec, so it works on any sandbox provider. Close cancels an activation still in flight, and the runner's shell records its PID and honors a stop marker, so an interrupted activation, or one that exceeds sandbox_runner_timeout_seconds, stops the runner. Tearing down the sandbox remains the backstop.

Sessions use MCP tool grants. Each activation adds the granted servers to that session's Hermes configuration, and the runner fails before the first model call when a required server does not connect. The sandbox installs Hermes with its mcp extra, which carries the MCP client. Hermes names MCP tools mcp_<server>_<tool>; the response reports them as mcp__<server>__<tool>, the form Gym strips before verification, while captured model calls keep Hermes' names. Other required grants, such as direct HTTP, are rejected at seed, and an MCP grant may not reuse a Hermes toolset name.

Single-agent-turn interaction flow

This diagram describes Hermes under the built-in single-agent-turn Environment Server. Other Environment Servers may use different Agent activation and sandbox ownership rules.

sequenceDiagram
    participant E as EnvironmentServer
    participant A as HermesAgent
    participant S as TaskSandbox
    participant M as ModelServer

    E->>A: Seed with optional SandboxAccess
    A->>S: Borrow or create sandbox
    E->>A: One /v1/responses activation
    A->>S: Run Hermes and tools
    S->>M: Model calls
    A-->>E: Agent response
    E->>A: Close session
    A->>S: Disconnect borrowed or stop owned sandbox

Loading

Stack position

Part 1 of 3, based on main. Next: #3721.

No issue is linked because this is one layer of the Environment Server stack.

Validation

• responses_api_agents/hermes_agent/.venv/bin/pytest responses_api_agents/hermes_agent/tests/ -q — 72 passed
• The runner tests run the pinned Hermes against a local HTTP model server and check that the iteration-limit summary and default OpenAI clients reach it, and that delegated children do not stream.
• Required Direct HTTP tool-grant reproduction now fails seed explicitly instead of running without the tool.
• A runner test starts the example MCP weather Resources Server with MCP exposure, grants it to Hermes, and checks that the model's tool call reaches the seeded Resources session; another checks that an unreachable required server fails before any model call.
• Non-Hermes real-model smoke: not run because this workspace has no configured model endpoint or credentials.

Rollouts

MCP grants: all 5 example_mcp_weather tasks with gpt-5.6-sol through single_agent_turn_legacy with the MCP tool transport scored reward 1.0, with Hermes calling get_weather over MCP from its sandbox. That run also used the Resources Server change that returns MCP access in resources_tools for Environment Server seeds, which is a separate PR. See the final integration PR for SWE-bench Pro rollout evidence.

Compatibility and security impact

reasoning_content replay on assistant messages comes from #3542; this PR adds tests for it. Sandbox requirements, including Model Server reachability, and the writable result-file limitation are documented in the Hermes README. Sandbox mode is suitable for evaluation, not RL training data, until result transport is isolated from model-written files.

[yaoyu-33]

This PR is okay. I have few other fixes in my PR later #3554

[yaoyu-33]

LGTM. Reviewed 6420620; 73 targeted tests passed, including the new MCP integration checks. Budget-exhaustion handling and further lifecycle fixes will follow in #3554. Preserving observations on runner exceptions remains a non-blocking follow-up.

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/36643794124