Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -656,6 +656,12 @@ include = [
# SWE harnesses create mutable runtime trees inside their package directory.
# Package discovery must never race or ship those caches and rollout outputs.
exclude = [
# Sandboxed harness agents keep one results directory per rollout; a long run leaves tens of
# thousands, and walking them stalls every package build.
"responses_api_agents.claude_code_sandboxed_agent.results",
"responses_api_agents.claude_code_sandboxed_agent.results.*",
"responses_api_agents.opencode_sandboxed_agent.results",
"responses_api_agents.opencode_sandboxed_agent.results.*",
"responses_api_agents.swe_agents.results",
"responses_api_agents.swe_agents.results.*",
"responses_api_agents.swe_agents.swe_openhands_setup",
Expand Down
2 changes: 1 addition & 1 deletion resources_servers/scale_swe/configs/scale_swe.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
scale_swe_resources_server:
resources_servers:
scale_swe:
allowed_agents: [opencode_sandboxed_agent]
allowed_agents: [opencode_sandboxed_agent, claude_code_sandboxed_agent]
entrypoint: app.py
domain: coding
verified: false
Expand Down
39 changes: 39 additions & 0 deletions resources_servers/scale_swe/configs/scale_swe_claude_code.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Wires the Claude Code sandboxed agent to scale_swe_resources_server, as
# scale_swe_opencode.yaml does for the OpenCode agent. Rows route here with
# agent_ref.name = scale_swe_claude_code_sandboxed_agent.
#
# gym env start --config resources_servers/scale_swe/configs/scale_swe_claude_code.yaml

config_paths:
- responses_api_agents/claude_code_sandboxed_agent/configs/claude_code_sandboxed_agent.yaml
- resources_servers/scale_swe/configs/scale_swe.yaml

scale_swe_claude_code_sandboxed_agent:
_inherit_from: claude_code_sandboxed_agent
responses_api_agents:
claude_code_sandboxed_agent:
resources_server:
type: resources_servers
name: scale_swe_resources_server
datasets:
- name: scale_swe
type: train
jsonl_fpath: resources_servers/scale_swe/data/scale_swe_training.jsonl
num_repeats: 1
# AweAI-Team/Scale-SWE has no declared license on the Hub (no license tag, empty card
# data) -- TBD rather than guessing one of the enum's real licenses.
license: TBD

# Rollout collection reaches an agent through its environment server. Named after the agent, not
# the environment: scale_swe_opencode.yaml's scale_swe_environment_server fronts the OpenCode
# agent, and a run loads both.
scale_swe_claude_code_environment_server:
environment_servers:
legacy_agent:
entrypoint: app.py
agent_server:
type: responses_api_agents
name: scale_swe_claude_code_sandboxed_agent
2 changes: 1 addition & 1 deletion resources_servers/swe_next/configs/swe_next.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
swe_next_resources_server:
resources_servers:
swe_next:
allowed_agents: [opencode_sandboxed_agent]
allowed_agents: [opencode_sandboxed_agent, claude_code_sandboxed_agent]
entrypoint: app.py
domain: coding
verified: false
Expand Down
37 changes: 37 additions & 0 deletions resources_servers/swe_next/configs/swe_next_claude_code.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Wires the Claude Code sandboxed agent to swe_next_resources_server, as
# swe_next_opencode.yaml does for the OpenCode agent. Rows route here with
# agent_ref.name = swe_next_claude_code_sandboxed_agent.
#
# gym env start --config resources_servers/swe_next/configs/swe_next_claude_code.yaml

config_paths:
- responses_api_agents/claude_code_sandboxed_agent/configs/claude_code_sandboxed_agent.yaml
- resources_servers/swe_next/configs/swe_next.yaml

swe_next_claude_code_sandboxed_agent:
_inherit_from: claude_code_sandboxed_agent
responses_api_agents:
claude_code_sandboxed_agent:
resources_server:
type: resources_servers
name: swe_next_resources_server
datasets:
- name: swe_next
type: train
jsonl_fpath: resources_servers/swe_next/data/swe_next_training.jsonl
num_repeats: 1
license: MIT

# Rollout collection reaches an agent through its environment server. Named after the agent, not
# the environment: swe_next_opencode.yaml's swe_next_environment_server fronts the OpenCode
# agent, and a run loads both.
swe_next_claude_code_environment_server:
environment_servers:
legacy_agent:
entrypoint: app.py
agent_server:
type: responses_api_agents
name: swe_next_claude_code_sandboxed_agent
2 changes: 1 addition & 1 deletion resources_servers/swe_rebench/configs/swe_rebench.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
swe_rebench_resources_server:
resources_servers:
swe_rebench:
allowed_agents: [opencode_sandboxed_agent]
allowed_agents: [opencode_sandboxed_agent, claude_code_sandboxed_agent]
entrypoint: app.py
domain: coding
verified: false
Expand Down
38 changes: 38 additions & 0 deletions resources_servers/swe_rebench/configs/swe_rebench_claude_code.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Wires the Claude Code sandboxed agent to swe_rebench_resources_server, as
# swe_rebench_opencode.yaml does for the OpenCode agent. Rows route here with
# agent_ref.name = swe_rebench_claude_code_sandboxed_agent.
#
# gym env start --config resources_servers/swe_rebench/configs/swe_rebench_claude_code.yaml

config_paths:
- responses_api_agents/claude_code_sandboxed_agent/configs/claude_code_sandboxed_agent.yaml
- resources_servers/swe_rebench/configs/swe_rebench.yaml

swe_rebench_claude_code_sandboxed_agent:
_inherit_from: claude_code_sandboxed_agent
responses_api_agents:
claude_code_sandboxed_agent:
resources_server:
type: resources_servers
name: swe_rebench_resources_server
datasets:
- name: swe_rebench_v2
type: train
jsonl_fpath: resources_servers/swe_rebench/data/swe_rebench_training.jsonl
num_repeats: 1
# nebius/SWE-rebench-V2's own Hub metadata license (cc-by-4.0).
license: Creative Commons Attribution 4.0 International

# Rollout collection reaches an agent through its environment server. Named after the agent, not
# the environment: swe_rebench_opencode.yaml's swe_rebench_environment_server fronts the OpenCode
# agent, and a run loads both.
swe_rebench_claude_code_environment_server:
environment_servers:
legacy_agent:
entrypoint: app.py
agent_server:
type: responses_api_agents
name: swe_rebench_claude_code_sandboxed_agent
2 changes: 1 addition & 1 deletion resources_servers/swemer_v1/configs/swemer_v1.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
swemer_v1_resources_server:
resources_servers:
swemer_v1:
allowed_agents: [opencode_sandboxed_agent]
allowed_agents: [opencode_sandboxed_agent, claude_code_sandboxed_agent]
entrypoint: app.py
domain: coding
verified: false
Expand Down
37 changes: 37 additions & 0 deletions resources_servers/swemer_v1/configs/swemer_v1_claude_code.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Wires the Claude Code sandboxed agent to swemer_v1_resources_server, as
# swemer_v1_opencode.yaml does for the OpenCode agent. Rows route here with
# agent_ref.name = swemer_v1_claude_code_sandboxed_agent.
#
# gym env start --config resources_servers/swemer_v1/configs/swemer_v1_claude_code.yaml

config_paths:
- responses_api_agents/claude_code_sandboxed_agent/configs/claude_code_sandboxed_agent.yaml
- resources_servers/swemer_v1/configs/swemer_v1.yaml

swemer_v1_claude_code_sandboxed_agent:
_inherit_from: claude_code_sandboxed_agent
responses_api_agents:
claude_code_sandboxed_agent:
resources_server:
type: resources_servers
name: swemer_v1_resources_server
datasets:
- name: swemer_v1
type: train
jsonl_fpath: resources_servers/swemer_v1/data/swemer_v1_training.jsonl
num_repeats: 1
license: NVIDIA Internal Use Only, Do Not Distribute

# Rollout collection reaches an agent through its environment server. Named after the agent, not
# the environment: swemer_v1_opencode.yaml's swemer_v1_environment_server fronts the OpenCode
# agent, and a run loads both.
swemer_v1_claude_code_environment_server:
environment_servers:
legacy_agent:
entrypoint: app.py
agent_server:
type: responses_api_agents
name: swemer_v1_claude_code_sandboxed_agent
2 changes: 1 addition & 1 deletion resources_servers/swemer_v2/configs/swemer_v2.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
swemer_v2_resources_server:
resources_servers:
swemer_v2:
allowed_agents: [opencode_sandboxed_agent]
allowed_agents: [opencode_sandboxed_agent, claude_code_sandboxed_agent]
entrypoint: app.py
domain: coding
verified: false
Expand Down
37 changes: 37 additions & 0 deletions resources_servers/swemer_v2/configs/swemer_v2_claude_code.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Wires the Claude Code sandboxed agent to swemer_v2_resources_server, as
# swemer_v2_opencode.yaml does for the OpenCode agent. Rows route here with
# agent_ref.name = swemer_v2_claude_code_sandboxed_agent.
#
# gym env start --config resources_servers/swemer_v2/configs/swemer_v2_claude_code.yaml

config_paths:
- responses_api_agents/claude_code_sandboxed_agent/configs/claude_code_sandboxed_agent.yaml
- resources_servers/swemer_v2/configs/swemer_v2.yaml

swemer_v2_claude_code_sandboxed_agent:
_inherit_from: claude_code_sandboxed_agent
responses_api_agents:
claude_code_sandboxed_agent:
resources_server:
type: resources_servers
name: swemer_v2_resources_server
datasets:
- name: swemer_v2
type: train
jsonl_fpath: resources_servers/swemer_v2/data/swemer_v2_training.jsonl
num_repeats: 1
license: NVIDIA Internal Use Only, Do Not Distribute

# Rollout collection reaches an agent through its environment server. Named after the agent, not
# the environment: swemer_v2_opencode.yaml's swemer_v2_environment_server fronts the OpenCode
# agent, and a run loads both.
swemer_v2_claude_code_environment_server:
environment_servers:
legacy_agent:
entrypoint: app.py
agent_server:
type: responses_api_agents
name: swemer_v2_claude_code_sandboxed_agent
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
results
94 changes: 94 additions & 0 deletions responses_api_agents/claude_code_sandboxed_agent/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Claude Code Sandboxed Agent

Runs the Claude Code CLI (`claude -p`) inside the task's own sandbox, the way
`opencode_sandboxed_agent` runs OpenCode: the resources server seeds the sandbox, this agent
attaches to it, runs one command under `sandbox_timeout`, reads the transcript the CLI left behind,
calls `/verify` and stops the sandbox. Gym stays on the host; the sandbox needs only the binary.

Claude Code talks to the Gym model server's `/v1/messages` route. Every Gym model server serves
that route, so the agent works with any of them. With `token_id_capture: true` (the default) each
call carries the rollout's training-capture URL prefix, so training gets the exact sampled tokens.

```bash
gym env start \
--config responses_api_models/vllm_model/configs/vllm_model.yaml \
--config nemo_gym/sandbox/providers/opensandbox/configs/opensandbox.yaml \
--config resources_servers/swe_rebench/configs/swe_rebench_claude_code.yaml
```

Each SWE resources server that has an OpenCode wiring config (`<server>_opencode.yaml`) has a
Claude Code one next to it (`<server>_claude_code.yaml`) with the agent instance
`<server>_claude_code_sandboxed_agent` and the environment server
`<server>_claude_code_environment_server` that fronts it; rows route to it with
`agent_ref.name = <server>_claude_code_sandboxed_agent`. A run may load both wiring configs.

## Installing Claude Code in the sandbox

Stage the release binary where every sandbox can read it, for example on a read-only bucket
mount, and set `remote_claude_code_binary_path` (and `remote_claude_code_musl_binary_path` for
musl-based images). The agent copies it to a writable directory, makes it executable and checks
that it reports `claude_code_version`. Without a staged binary the official installer runs inside
the sandbox, which needs network access.

```bash
VERSION=2.1.287
curl -fsSL -o claude https://downloads.claude.ai/claude-code-releases/$VERSION/linux-x64/claude
curl -fsSL -o manifest.json https://downloads.claude.ai/claude-code-releases/$VERSION/manifest.json
grep -q "$(sha256sum claude | cut -d' ' -f1)" manifest.json && echo MATCH || echo MISMATCH
```

## What the agent sets

- A fresh `CLAUDE_CONFIG_DIR`, binary and transcript directory per rollout under
`remote_work_dir` (`/tmp`), outside the task repository; stdin closed.
- The full harness: `bare: false`. `--bare` is Claude Code's minimal mode, with a one-line system
prompt and only the Bash, Read and Edit tools.
- `--setting-sources user`: only the rollout's empty user settings and `--settings` load, so a task
repository's `.claude/` settings and hooks cannot change the run.
- No auto-memory (`CLAUDE_CODE_DISABLE_AUTO_MEMORY=1`): each rollout starts from an empty config
directory, and notes written mid-session would change the context of later calls.
- `--dangerously-skip-permissions` with `IS_SANDBOX=1`.
- `--disallowedTools` for what does not fit a one-shot run in a task sandbox: web access and skills
(as for the OpenCode agent), worktrees (they would move the edits out of the repository the
verifier diffs), scheduled prompts, wakeups, workflows, agent messaging and the code-review report
tool. Agent (subagents), Bash, Read, Edit, Write, NotebookEdit and TaskStop stay.
- Every model role (`ANTHROPIC_MODEL`, the opus/sonnet/haiku defaults, subagents) points at `model`.
- No update checks, telemetry, error reports, experimental betas or per-request attribution text.
The attribution text would change the system prompt on every call and break capture's call chain.
- `CLAUDE_STREAM_FIRST_BYTE_TIMEOUT_MS=1800000` and `API_TIMEOUT_MS=3600000`. A Gym model server
sends a streamed reply's headers only after generation finishes, and Claude Code's default wait
of 3 to 5 minutes for headers would abort and retry long calls.
- `auto_compact: false` sets `DISABLE_COMPACT`, `DISABLE_AUTO_COMPACT` and the `autoCompactEnabled`
and `precomputeCompactionEnabled` settings to off. Compaction rewrites the conversation, which
token capture cannot follow; training must turn it off. With it off, Claude Code sends requests
until the model server rejects one, like OpenCode with compaction off.
- The CLI runs under `timeout` until `harness_timeout_margin_s` before `sandbox_timeout`, so it
cannot keep editing the repository or calling the model while the verifier runs.

`claude_code_env` and `claude_code_settings` are layered over all of the above.

## Output

The stream-json output is downloaded to `results/<session>/stream.jsonl` (or under
`artifacts_dir`) and converted to Responses items: the main conversation's reasoning, text, tool
calls and tool results. Subagent traffic and Claude Code's own error notices are left out. A
`generation.json` receipt with the response and the execution fields is written before grading.
When observations are collected, the session transcripts are archived and read with
`claude_code_agent.observability`, which also reports any compaction.

The `/run` result carries, besides the verifier's fields:

- `claude_code_finished`, `claude_code_exit_code`, `claude_code_error_type`, `claude_code_failed`
(exec error, nonzero exit, missing transcript or error result), `claude_code_export_found`,
`claude_code_results_fpath`, `claude_code_run_stdout`, `claude_code_run_stderr`.
- How the session ended, from its final `result` event: `claude_code_result_success`,
`claude_code_result_error`, `claude_code_result_missing` (the deadline or a crash cut it off),
`claude_code_context_overflow` (the last request no longer fit the context), and
`claude_code_duration_s`.
- What it did, main conversation and subagents together: `claude_code_main_turns`,
`claude_code_subagent_turns`, `claude_code_subagent_calls`, `claude_code_tool_calls`,
`claude_code_tool_errors` (calls Claude Code refused) and `claude_code_unknown_tool_calls`
(refused for naming a tool that does not exist).

With `execution_failure_reward_zero: true` a failed run is graded as an empty response instead of
whatever patch it left behind; the verifier must score an empty response as an unmasked zero.
Empty file.
Loading
Loading