[hemildesai]

What changed and why

Follow-up to #2636 (sandbox pause/resume). On cell-3, re-attaching to a PTY session opened before a pause took about 95s to fail, and the error was misleading:

SandboxPtyError: PTY session already has an attached client

On the Kubernetes backend, pause commits the rootfs and replaces the runtime, so the old session no longer exists. But execd answers a WebSocket attach to a missing session with the same 1008 close it uses for a session held by another client. I checked this live: a random session id, a random id with takeover=1, and a genuinely held session all close 1008 with an empty reason within 0.3s. So Gym treated it as a stale holder and waited out every retry: the session's own takeover retries (2+5+10s), the provider's retries, and a final sandbox-death check.

execd's HTTP API does tell the two apart, which the same live probe showed:

Request	Answer
GET /pty/<missing>	404 {"code":"CONTEXT_NOT_FOUND","message":"pty session … not found"}
GET /pty/<live>	200 {"session_id":…,"running":true,…}

OpenSandboxProvider.attach_pty() now makes that GET before dialing. Only execd's own 404, with the CONTEXT_NOT_FOUND code, counts as missing. A proxy 404 while a route is still registering, or a failed check, lets the attach proceed exactly as before. A missing session now fails immediately with PTY session <id> not found, with or without takeover. Before, without takeover, a missing session was also misreported as "already has an attached client".

Cost: one extra small GET per attach. Attaches are reconnect operations, not per-command.

No issue filed. This was found while validating #2636 end to end.

Validation

Unit tests (test_opensandbox_pty.py, test_opensandbox_provider.py, test_sandbox.py): 246 passed. New cases cover:

• A missing session fails with "not found", makes no WebSocket dial, and never consults sandbox status. Covered for both takeover values.
• A proxy 404 without execd's code, and a failed check, both keep the existing takeover retries.

Two existing tests now count the existence-check client. pre-commit run on the changed files passes.

Live on cell-3 (tenant namespace, Kubernetes backend), running the pause/resume e2e across two OS processes:

Step	Result
pause, then reconnect from a new process	paused
resume	running
file written before pause	preserved
non-PTY exec, foreground path	rc 3; stdout and stderr correct
non-PTY exec, background path (what opensandbox.yaml enables)	rc 3; combined stream in stdout, as documented
attach to the pre-pause PTY session	fails in 0.3s with not found (was about 95s with "already has an attached client")
fresh PTY after resume	works

A separate probe confirmed the normal case is unaffected. Detaching from a live session and re-attaching by id succeeds with and without takeover, in about 0.6s, with shell state intact. All sandboxes were deleted afterwards.

Rollout evidence: N/A. This is a sandbox provider change with no environment, agent, or benchmark behavior change.

Noticed, not changed here

The foreground exec path appends the SDK's error to stderr, so a command that exits non-zero gets a synthetic CommandExecError: <code> line after its own stderr. This behavior dates from the original sandbox API, and it's out of scope here.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[hemildesai]

/ok to test d2cb1a7

[hemildesai]

The three red checks are unrelated to this change. Server suite (shard 3) fails only on responses_api_agents/osworld_agent, whose environment cannot resolve:

osworld==0.1.0 depends on numpy<2 … and you require osworld and numpy>=2.1,<2.5 … requirements are unsatisfiable

That conflict came in with #3592, whose server-suite shards were skipped. Nemo_CICD_Test and Server suite are the aggregates of that shard. The fix is #3649 by the OSWorld owner. Every other check passed, including Unit tests / Test, which runs the sandbox and PTY tests this PR touches. I'll re-run CI here once #3649 lands.

🤖 Generated with Claude Code

[hemildesai]

/ok to test 5af14a4

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/35881522695