Skip to content

feat(swe): serve typed resources sessions from SWEmer Ultra, SWEmer OML, and SWE internal v1 - #4303

Open
ananthsub wants to merge 2 commits into
mainfrom
ananthsub/swemer-typed-sessions
Open

ananthsub wants to merge 2 commits into
mainfrom
ananthsub/swemer-typed-sessions

Conversation

@ananthsub

@ananthsub ananthsub commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

What changed and why

This PR makes each server inherit from SandboxSessionResourcesServer, added in #4302 to the SWEmer Ultra (resources_servers/swemer_ultra, #4116), SWEmer OML (resources_servers/swemer_oml, #3799), and SWE internal v1 (resources_servers/swe_internal_v1, #3933) resources servers. Like swe_gym and r2e_gym before #4302, these servers accept only the legacy flat instance body on /seed_session, so an environment server's typed seed is rejected with a 422, and nothing releases the task sandbox when an episode ends without verify. After this PR, each server serves environment server episodes with the same behavior #4302 describes, while the legacy request and response stay unchanged.

The change to each server's app.py is the same:

  • The server inherits from SandboxSessionResourcesServer instead of SimpleResourcesServer.
  • seed_session accepts either body. A ResourcesSeedSessionRequest goes to seed_task_sandbox_session with the server's own instance model, and the legacy body follows the same steps as before.
  • The legacy seed steps move into _start_task_sandbox, which registers the sandbox as soon as it exists and returns the agent's working directory. The working directory is the row's own workdir.
  • _forget_task_sandbox_state drops the per-session state that seeding records (the untracked files the image ships and the base commit), so a close or an expired session releases it.
  • verify marks a typed session's sandbox as consumed before it captures the patch, so a repeated verify fails instead of grading an empty patch, and the stop at the end of patch capture goes through _release_task_sandbox, so a failed stop leaves the sandbox for close to retry.

Each server also gets tests/test_sessions.py, which runs a typed episode (seed, verify, repeated verify, close), a legacy seed and verify, and the session conformance check through the real server with a fake sandbox, using the first row of the server's data/example.jsonl.

Issue

No issue is linked. This continues #4302, which adds typed resources sessions to the SWE servers sdevare-nv added.

Stack position

Based on #4302; review that PR first. A sibling PR based on the same branch migrates the six SWE training servers from #3327.

Validation

  • NEMO_GYM_EXTRA_ROOTS=$PWD PYTHONPATH=$PWD python -m pytest resources_servers/swemer_ultra/tests resources_servers/swemer_oml/tests resources_servers/swe_internal_v1/tests resources_servers/swebench/tests/test_sandbox_sessions.py tests/unit_tests/test_session_contract_repo.py -q passed.
  • With each server's previous app.py, its tests that use a typed seed fail (the typed seed is rejected with a 422) and its legacy test passes.
  • pre-commit run on the changed files.

Rollouts

One SWEmer Ultra task (getmoto__moto-7365) ran end to end. The run used the same local configuration and setup as the rollouts in #4302: single_agent_turn_legacy with hermes_agent, Docker sandboxes with host networking, Gym's vLLM model server, Qwen3-4B-Instruct-2507 served by vLLM, and a system prompt that tells the model to edit files. These runs check the session plumbing, not solve rate. Hermes ran its terminal commands in the row's working directory inside the task sandbox it received through sandbox_access; it cloned an unrelated repository there instead of editing moto. verify captured exactly that change from the task sandbox (a 211-byte patch adding the clone as a submodule entry), applied it in a fresh sandbox, and completed grading with a reward of 0. The failure sidecar was empty, and no task sandbox was left running after the session closed. SWEmer OML and SWE internal v1 share this seed and verify path, and each is covered by the HTTP-level tests above; their example rows were not run.

These rollouts ran before the session code moved from a mixin into the SandboxSessionResourcesServer base class and before the operate scope was added. Neither change alters what runs on Docker: the methods are the same, and the Docker provider ignores the scope.

Compatibility

  • Legacy seed and verify bodies and responses are unchanged, so opencode_sandboxed_agent and existing configurations keep working. A legacy seed whose preparation fails now leaves its sandbox registered for shutdown instead of unreferenced.
  • A typed session's repeated verify returns an error instead of a reward of 0.
  • No benchmark results change: grading is untouched.

@ananthsub ananthsub added feature New capabilities, enhancements, or enablement work area:environment Individual environments, benchmarks, verifiers, and environment-specific resources servers needs-review PR is ready for code review and waiting on a reviewer labels Oct 9, 2026
@nemo-automation-bot
nemo-automation-bot Bot requested a review from a team October 9, 2026 16:50
Add resources_servers/swebench/task_sandbox_sessions.py, a shared helper for
SWE resources servers whose seed starts one task sandbox, and use it in
swe_gym and r2e_gym. An Environment Server can now seed a typed session,
hand the agent the task sandbox through sandbox_access, verify, and close.

The legacy seed body and response are unchanged.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
…ML, and SWE internal v1

Use TaskSandboxSessions in swemer_ultra, swemer_oml, and swe_internal_v1, so
an Environment Server can seed a typed session, hand the agent the task
sandbox through sandbox_access, verify, and close.

The legacy seed body and response are unchanged.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
@ananthsub
ananthsub force-pushed the ananthsub/swe-typed-sessions branch from e8a8665 to 652d384 Compare October 9, 2026 17:13
@ananthsub
ananthsub force-pushed the ananthsub/swemer-typed-sessions branch from 886e260 to 7e97fda Compare October 9, 2026 17:13
@yaoyu-33 yaoyu-33 added the complexity:medium Single-domain change with interacting parts or a moderate review surface label Oct 9, 2026
@github-actions github-actions Bot added the sla:review-overdue Review response is over the one-business-day SLA label Oct 12, 2026
Base automatically changed from ananthsub/swe-typed-sessions to main November 5, 2026 02:37
@nemo-automation-bot
nemo-automation-bot Bot requested a review from a team November 6, 2026 22:07

This branch was previously deployed

1 inactive deployment
public — 7e97fda0 Deployed Oct 9, 2026 by copy-pr-bot[bot] via release / finalize / notify #4875
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:environment Individual environments, benchmarks, verifiers, and environment-specific resources servers complexity:medium Single-domain change with interacting parts or a moderate review surface feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer sla:review-overdue Review response is over the one-business-day SLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants