Skip to content

fix(anyswe_agent): keep the runtime bundle off the agent PATH and isolate the runner Python - #4018

Merged
linj-glitch merged 1 commit into
NVIDIA-NeMo:mainfrom
linj-glitch:linj/anyswe-runtime-off-path
Oct 13, 2026
Merged

linj-glitch merged 1 commit into
NVIDIA-NeMo:mainfrom
linj-glitch:linj/anyswe-runtime-off-path

Conversation

@linj-glitch

Copy link
Copy Markdown
Collaborator

What does this PR do?

anyswe_agent runs its harness from a portable runtime bundle (CPython, NeMo Gym, Node and the CLI) inside the task image. Two leaks between that bundle and the task:

1. The agent's python, pip and node were the bundle's. agent_runner.py prepends the whole bundle bin/ to PATH, and that directory also holds python3, pip, node and npm. Every shell command the coding agent runs therefore resolves python / pip / node to the bundle instead of the task image. Agents working on numpy, pandas or Pillow tasks then import, "fix" or reinstall the bundle's copies instead of the task's. When the bundle is shared, for example a baked or bind-mounted runtime, one rollout's writes break every later sandbox.

For the CLI harnesses (claude, cline, opencode, openclaw, pi), the runner now symlinks only the harness executable into a private temp directory and puts that first on PATH. If the executable is a node script (#!/usr/bin/env node, as npm-installed CLIs are), node is linked too so the CLI still starts. Python harnesses (Hermes, NeMo Fabric) and anything unknown keep the whole bundle bin/ as before.

2. The task image's Python environment could break the runner. The runner was started with plain <runtime>/bin/python. If the task image sets PYTHONPATH (or has a user site) pointing at a checkout of numpy or pandas, the runner imports that source tree instead of the runtime's wheel and dies before its first model call ("Unable to import required dependency numpy"). It now starts with python -I. The runner already adds /nemo_gym_mount to sys.path itself, and child processes the agent spawns still get the normal environment.

The README documents both.

Validation

  • python -m pytest -q responses_api_agents/anyswe_agent/tests: 24 passed.
  • New TestRuntimeIsolation in tests/test_runtime_isolation.py:
    • test_unknown_harness_keeps_the_whole_bundle_bin
    • test_native_cli_hides_the_bundle_python_pip_and_node: the shim holds only claude.
    • test_node_script_cli_also_gets_node: the shim holds claude and node.
    • test_runner_python_ignores_the_task_images_python_environment: drives _run_agent_in_sandbox with a fake sandbox and checks the runner command uses python -I.
  • pre-commit run --files <touched files>: all hooks pass.

Rollouts

Both problems showed up in internal GRPO training of Claude Code on R2E-Gym / SWE task images. With the shared runtime on a cluster filesystem, an agent overwrote a file inside the bundle's numpy, and every later rollout of that run failed and was masked. In numpy/pandas images the runner failed at import. With these changes the training ran several thousand rollouts without either failure. I have no public rollout numbers to share.

Compatibility and benchmark impact

  • With the CLI harnesses, the agent's shell now sees the task image's python / pip / node rather than the bundle's. Tasks that need Python or Node must get them from the image, which is the intended environment. If an image has no python at all, the agent no longer silently gets CPython 3.13 from the bundle.
  • The Hermes and NeMo Fabric harnesses are unchanged.
  • The runner ignores PYTHON* environment variables of the image.

Checklist

  • I have read the contributing guidelines.
  • The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
  • Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable).
  • Documentation added or updated, or N/A with justification.
  • Pre-commit checks pass locally (pre-commit run --all-files) (so CI lint/format checks pass). (Run on the touched files.)
  • New source files include the required Apache-2.0 SPDX header. (N/A: no new files.)
  • All commits have DCO sign-off (git commit -s) (so the DCO check passes).

🤖 Generated with Claude Code

@copy-pr-bot

copy-pr-bot Bot commented Oct 4, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@yaoyu-33 yaoyu-33 added area:agent Agent harnesses and Responses API agent behavior bug Something isn't working complexity:medium Single-domain change with interacting parts or a moderate review surface needs-review PR is ready for code review and waiting on a reviewer labels Oct 5, 2026
@linj-glitch
linj-glitch marked this pull request as draft October 5, 2026 06:11
@linj-glitch
linj-glitch force-pushed the linj/anyswe-runtime-off-path branch from bc19465 to 666d426 Compare October 8, 2026 00:15
…late the runner Python

The in-sandbox runner prepended the whole runtime bundle `bin/` to PATH. That
directory also holds the bundle's python3, pip, node and npm, so the coding
agent's `python`, `pip` and `node` resolved to the bundle instead of the task
image. Agents solving numpy/pandas/Pillow tasks imported, "fixed" or
reinstalled the bundle's copies instead of the task's. When the bundle is
shared (a baked or bind-mounted runtime), one rollout's writes break every
later sandbox.

For the CLI harnesses (claude, cline, opencode, openclaw, pi), link only the
harness executable into a private directory and put that first on PATH. Add
`node` too when the executable is a node script. Other harnesses keep the
whole bundle `bin/`.

Start the runner with `python -I` so the task image's PYTHONPATH and user
site cannot shadow the runtime's own packages. In images whose checkout is
numpy or pandas, the runner otherwise dies importing nemo_gym before it
makes a single model call.

Signed-off-by: Lin Jia <linj@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linj-glitch
linj-glitch force-pushed the linj/anyswe-runtime-off-path branch from 666d426 to ff5daeb Compare October 8, 2026 00:20
@linj-glitch
linj-glitch marked this pull request as ready for review October 8, 2026 00:30
@linj-glitch

Copy link
Copy Markdown
Collaborator Author

/ok to test ff5daeb

@nemo-automation-bot
nemo-automation-bot Bot requested a review from a team October 8, 2026 00:38
@linj-glitch
linj-glitch removed the request for review from ananthsub October 8, 2026 01:03
@github-actions github-actions Bot added the sla:review-overdue Review response is over the one-business-day SLA label Oct 9, 2026

@alexandery-nvidia alexandery-nvidia left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I traced the -I interplay before approving: isolated mode drops the script directory, PYTHONPATH, and the user site, which is safe here because stdlib and fastapi resolve from the bundle interpreter's own site-packages (the deps scripts install nemo_gym with dependencies into the bundle), and the deferred main() imports resolve through the explicit /nemo_gym_mount insert, which also shadows the bundle's own nemo_gym copy; nothing imports a sibling of the script. The shim fallback (unknown harness keeps the whole bin/) preserves current behavior for out-of-catalog agents, and the node-shebang detection covers the CLIs that are node scripts. Ran tests/test_runtime_isolation.py (4 passed) and tests/test_app.py (20 passed) at the PR head locally.

@github-actions github-actions Bot removed the sla:review-overdue Review response is over the one-business-day SLA label Oct 12, 2026
@linj-glitch
linj-glitch added this pull request to the merge queue Oct 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38327143475

Merged via the queue into NVIDIA-NeMo:main with commit 8fddfb5 Oct 13, 2026
36 of 37 checks passed
@linj-glitch
linj-glitch deleted the linj/anyswe-runtime-off-path branch October 13, 2026 00:42

This branch was previously deployed

1 inactive deployment
public — ff5daeb2 Deployed Oct 8, 2026 by copy-pr-bot[bot] via release / finalize / notify #4644
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent harnesses and Responses API agent behavior bug Something isn't working complexity:medium Single-domain change with interacting parts or a moderate review surface needs-review PR is ready for code review and waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants