[martin-nv]

What does this PR do?

Adds responses_api_agents/pool_sandboxed_agent, a harness that runs Poolside's pool CLI coding agent headlessly (pool exec) inside the resources-server-owned task sandbox, following the OpenCode sandboxed agent pattern: the SWE-bench resources server creates the task sandbox, the agent installs pool into it, points it at Gym's per-rollout model URL, runs the task, and leaves the edits in the workdir for the resources server to diff and grade.

• pool is configured through a generated agent config file (--agent-config-file) mirroring its CLI defaults, with an OpenAI-compatible provider aimed at the Gym proxy, use_streaming: false (token ids and logprobs are only returned on non-streaming responses), and compaction thresholds derived from pool_max_context_window. pool_agent_config deep-merges overrides.
• HOME and the XDG dirs are redirected under /tmp so pool's config, state and trajectories never land in the repo workdir that gets diffed. Pool's own container sandboxing is disabled and tool approvals are auto-allowed for the non-interactive run.
• The pool exec -o json event stream is converted into Responses output items (messages with <think> reasoning, function_call / function_call_output pairs). Usage is not part of the stream, so usage is empty for now.
• Benchmark wiring for SWE-bench Verified, Multilingual, Pro, and Terminal Bench 2.1. Includes the Lightning SWE Verified profile and Laguna XS profiles for all four benchmarks. XS SWE Pro defaults to two nodes with concurrency 128 * NUM_NODES (256 by default); XS Multilingual and Terminal Bench use concurrency 128.
• nemo_gym/openai_utils.py: pool attaches Anthropic-style cache_control: {type: ephemeral} hints to the last content part and tool spec of every chat request, which the strict params models rejected with 422. The hint is now typed (NotRequired[CacheControlEphemeralParam]) on the chat text content part and tool param.

Stacked on #3602 (codex/benchmark-fast-runs); review this diff against that branch.

Validation

• New XS Pool profiles: Gym offline config resolution, agent/resource references, Bash syntax, and SWE Pro concurrency checks passed. Full pre-commit and secret scans passed. No additional live rollout was launched for Multilingual, Pro, or Terminal Bench; those combinations still need end-to-end evaluation.

• pytest responses_api_agents/pool_sandboxed_agent/tests tests/unit_tests/test_openai_utils.py and pre-commit run on the changed files.

• Two one-task smoke runs of SWE-bench Verified (astropy__astropy-12907) on OpenSandbox via gym eval run ... --limit 1, with the policy model served through an OpenAI-compatible endpoint (nemotron-3-ultra). Both resolved with reward 1.0: pool installed from the public installer inside the sandbox, reached the Gym proxy, produced a 13-line patch, and the resources server graded it with 15/15 tests passing. A third one-task run through the Slurm launcher on Laguna XS 2.1 (vLLM, DFlash) also resolved; a full 1500-rollout SWE-bench Verified run on that model is in progress.

Rollouts

Smoke rollout transcript (streaming run) had 150 output items; the non-streaming run 132. Available on request; not attached since they contain the full task prompt and model reasoning.

Compatibility and benchmark impact

No change to existing harnesses. Typing cache_control widens the accepted chat schema; the field is forwarded to the backend, which vLLM accepts (extra="allow"). No anti-cheat tool restrictions are configured for pool yet, unlike the OpenCode deny-list; SWE-bench Verified repos are already clean, and pool exposes no equivalent permission config today.

Checklist

• I have read the contributing guidelines.
• The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
• Tests added or updated and pass locally.
• Documentation added or updated (responses_api_agents/pool_sandboxed_agent/README.md).
• Pre-commit checks pass locally (pre-commit run on the changed files).
• New source files include the required Apache-2.0 SPDX header.
• All commits have DCO sign-off (git commit -s).

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[christian-muertz]

It seems that pool is sending non standard cache_control. Instead of stripping I would rather add it to the type, wdyt?

[martin-nv]

Agreed. I guess it would crash on a vllm endpoint that doesn't support the extra field, but maybe better like this

[martin-nv]

Done in f801bc9: dropped the strip and typed cache_control as NotRequired[CacheControlEphemeralParam] on NeMoGymChatCompletionContentPartTextParam and NeMoGymChatCompletionToolParam (the tool-message part reuses the text part). Note the field now flows through to the backend via model_dump(exclude_unset=True); vLLM's request models are extra="allow" so that's fine there, but the OpenAI backend would reject it inside content parts.

[martin-nv]

Went back and forth on this with Martin. We'd rather not widen the shared chat schema in openai_utils.py for an Anthropic-only hint no Gym backend acts on, so the latest commit drops it at the chat dispatch instead, now targeted to messages[].content[].cache_control and tools[].cache_control only (a cache_control key inside a tool's JSON schema is untouched). Happy to revisit if you feel strongly.