[hemildesai]

What does this PR do?

Add optional automatic pause and traffic-triggered resume support to Gym's E2B provider for AgentENV. A sandbox that reaches its TTL can retain its state and resume when the agent next uses it, instead of becoming unusable until manually resumed or deleted.

• Add e2b.create.auto_resume, default false: false explicitly sends kill-on-timeout plus disabled auto-resume; true sends pause-on-timeout plus auto-resume; null sends no lifecycle and leaves the action to the SDK/service.
• Why false by default: E2B 2.36–2.41 default to kill-on-timeout, but 2.42+ (including the locked 2.46) omit the lifecycle fields. AgentENV then pauses timed-out sandboxes without auto-resume and never evicts them, so a crashed or cancelled run leaks paused VMs. We hit this in practice: sandboxes from a crashed client stayed paused for hours past their TTL. This folds in fix(sandbox): kill E2B sandboxes at timeout instead of pausing them #4598.
• Read control-plane state for status checks, reporting PAUSED without sending an envd health request that would wake the VM.
• Cover lifecycle serialization with the real SDK and document timeout and cleanup behavior.

No separate issue is needed for this focused integration of AgentENV's existing auto-eviction and proxy auto-resume APIs.

Validation

• python -m pytest tests/unit_tests/test_e2b_provider.py tests/unit_tests/test_e2b_build.py tests/unit_tests/test_sandbox.py tests/unit_tests/test_sandbox_connect.py tests/unit_tests/test_sandbox_access.py --no-cov -q -o addopts='': 221 passed, 2 skipped, with locked E2B 2.46.0. The skips are existing opt-in sandbox survival tests.
• python -m pytest tests/unit_tests/test_e2b_provider.py --no-cov -q -o addopts='' with minimum E2B 2.36.0: 112 passed.
• Regression check: the six new lifecycle/status cases fail against the unmodified provider on main and pass with this change.
• pre-commit run --all-files: passed.
• cd fern && npm run check -- --warnings: zero errors; missing-redirect checks skipped because Fern authentication is unavailable.
• git diff --check: passed.

CI classification is full because this changes the shared sandbox provider. The complete core/server suites were not run locally.

Follow-up commit (kill by default, review fixes):

• test_e2b_provider.py + test_e2b_build.py + sandbox tests: 224 passed, 2 skipped with locked E2B 2.46.0; test_e2b_provider.py 115 passed with E2B 2.36.0.
• New test_create_kills_at_timeout_by_default fails with the old None default; new test_real_sdk_status_only_reads_control_plane asserts that status sends only GET /sandboxes/{id} through the real SDK (2.36.0 and 2.46.0).
• pre-commit run --all-files and fern check --warnings: passed (0 errors; the redirect check is skipped without Fern auth).

Rollouts

Live provider smoke passed with both E2B 2.36.0 and locked E2B 2.46.0 on an AgentENV deployment, using the osworld-slim template and certificate-verified HTTPS with test-local CA trust:

• Created a sandbox with a 30-second TTL, wrote a marker file, and executed a command.
• Observed automatic PAUSED state at expiry; repeated status checks did not wake it.
• Executed a command through the same handle, automatically resuming the VM and reading the preserved marker (0.422 seconds for this one request; not a performance benchmark).
• Expired it again and verified a file read also resumed it with contents intact.
• Verified auto_resume: false deletes at expiry (control-plane 404) rather than retaining a paused VM.
• Explicitly cleaned up test resources.
• Kill-on-timeout default: two sandboxes with a 60 s TTL (one with a sandbank opt-in, one plain), created through the provider with lifecycle={"on_timeout": "kill"} on AgentENV and e2b 2.53.1, were gone at 95 s (GET 404, not in the paused listing). Before the change, same-shape sandboxes were still listed as paused hours after their TTL.

Compatibility and benchmark impact

Behavior change: sandboxes are now deleted at timeout by default on every SDK version, which matches E2B's documented default and the 2.36–2.41 behavior. Only E2B 2.42+ against a server that defaults to pausing (AgentENV) changes, from a never-reaped pause to deletion. Set auto_resume: true to opt into pause and resume, or null to restore the previous "send nothing" behavior. No benchmark profile is changed. The new dataclass parameter is keyword-only to preserve positional compatibility. E2B status now reflects control-plane state rather than guest-health reachability.

TTL remains a lifetime deadline, not an idle timer. AgentENV currently imposes a server-side EnsureMinimum(300s) after proxy auto-resume; this PR does not change that upstream floor. Paused state retains storage, so normal cleanup is still required. No dataset, verifier, or scoring changes; benchmark-result impact has not been measured.

Checklist

• I have read the contributing guidelines.
• The change is focused; unrelated edits are excluded.
• Tests added or updated and pass locally.
• Documentation added or updated.
• Pre-commit checks pass locally.
• New source files include the required SPDX header (no new source files).
• All commits have DCO sign-off.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-3757-050b9394c8c7.docs.buildwithfern.com/nemo/gym

[hemildesai]

/ok to test 8daf471

[hemildesai]

/claude review

[hemildesai]

/ok to test b1d98d0

[hemildesai]

/claude review

[claude]

SHIP — low-risk, well-scoped E2B lifecycle change.

Reviewed the full diff across the four changed files. Summary of what I checked and one non-blocking note.

What is correct:

• auto_resume defaults to None, and create() only emits lifecycle when it is non-None. Existing deployments see no behavior change, and the YAML exemplar carries the default (auto_resume: null) rather than a Python-side default — consistent with the config convention.
• The pause/kill coupling (on_timeout: "pause"|"kill" derived from the same bool) means resume cannot be enabled without pause, so a sandbox cannot be left paused indefinitely with resume off.
• status() switching is_running() -> get_info() correctly avoids waking a paused VM with a data-plane health check, still routes not-found -> STOPPED, and maps unrecognized states -> UNKNOWN (verified for "resuming"). States other than running/paused degrade safely rather than mislabeling.
• get_info uses _request_params() (request-timeout only), matching the data-plane contract; the updated docstring is accurate.
• Test coverage is solid: lifecycle policy serialization (autoPause/autoResume wire fields via real-SDK capture), non-boolean rejection, and the three-way state->status mapping including the "must not send traffic that resumes" guard.

NOTE (author's call, defense-in-depth): The status mapping keys on the literal strings "running"/"paused" from info.state. The mocked FakeSandbox.get_info returns these lowercase strings, but the real-SDK test (test_real_sdk_user_agent_and_call_shapes) only binds the get_info signature — it never asserts the actual attribute name or value that E2B's SandboxInfo returns. If a future SDK within the >=2.36,<3 range represents state differently (enum, casing), every live sandbox would silently map to UNKNOWN while unit tests stay green. A one-line assert on a real SandboxInfo.state value (guarded by the existing importorskip) would close that gap. Not blocking — the current SDK uses these literals.

Ran the pinned base/head revision check before and after analysis; both matched b1d98d0.

[hemildesai]

/claude review

[hemildesai]

/ok to test ba72878

[svcnvidia-nemo-ci]

SHIP: LGTM. I found nothing wrong in the provider change itself. I'm leaving two non-blocking NOTEs on the new real-SDK tests.

What I checked:

• Config and validation. YAML and mapping values for create.auto_resume (default false) reach the dataclass. Non-boolean values are rejected in E2BCreateConfig.__post_init__.
• Create paths. create() and the REST path in _create_with_extension_params send the same lifecycle for true and false, and neither sends one for null.
• Status. status() maps running, paused, unrecognized and transient-failure cases correctly. The real-SDK test confirms it sends only the control-plane GET.
• Status consumers. The sandbox facade passes status through unchanged. OSWorld's readiness loop keeps polling on PAUSED or UNKNOWN, so neither state makes a live sandbox look dead.
• In-repo AgentENV profile. osworld_agentenv.yaml doesn't set auto_resume, so its sandboxes are now deleted when the TTL expires. That's the leak fix the PR describes, and the compatibility notes cover it.
• Docs. The docs page, the provider README and the e2b.yaml comments match the head code.
• Earlier threads. The current head addresses the earlier review threads: the default value, docs consistency, the resume floor, the 410 behavior, connect resetting the timeout, and the real-SDK status test.

One behavior change to keep in mind: if the control plane reports a sandbox as running but its envd is dead, status() now returns RUNNING where it used to return STOPPED. In that case OSWorld readiness waits out its own timeout instead of failing early. The PR states this, so I'm not counting it as a finding.

On rollouts, the description still lists a model-driven rollout as pending. AGENTS.md's smoke-rollout rule is aimed at environment and agent changes, and the live AgentENV smoke run exercised the changed create, status and close paths, so I'm not treating the pending rollout as a merge gate.

One of my three review passes (docs accuracy) finished without its report reaching me, so I checked the docs page, README and YAML comments against the code myself.

Review generated by claude. This was a full review. Request a full review with /review reset=true.

[hemildesai]

/ok to test 050b939

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41531613587

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41534673535

[ananthsub]

Now that pyproject.toml requires e2b>=2.46.0 (since #4822), null always means "send no lifecycle fields". On hosted E2B that behaves like false. On AgentENV it is the paused-and-never-reaped case this PR fixes. Is there a gateway that rejects autoPause/autoResume and needs null?

• If not, a plain bool removes the footgun, in line with CLAUDE.md's "do not add options for hypothetical future needs".
• If it stays:
  • give that reason in the null row of the docs table (e2b.mdx#L229-L231).
  • add None to test_real_sdk_serializes_lifecycle_policy and assert that autoPause and autoResume are absent from the request body. Today, a change that makes the REST create path send the kill fields for null still passes every test.

Either way, the 2.36–2.41 history here, in e2b.yaml#L67-L68 and in e2b.mdx#L233-L235 describes versions the floor no longer allows. "the locked 2.46" will also go stale with the next lock bump. Saying that omitting the fields leaves the choice to the server (AgentENV pauses without resume) should be enough. A related, older line at e2b.mdx#L24 still says e2b>=2.36.0.

[ananthsub]

The new one-liner no longer holds for every caller. status() now passes these params to get_info, a control-plane call that accepts the full ApiParams (e2b 2.46 sandbox_async/main.py#L662-L675). It also drops the reason the helper exists. Suggest keeping the old explanation and adding get_info:

Suggested change

	"""Return the request timeout; data-plane methods reject connection parameters."""
	"""Per-request options for calls on an existing sandbox object.
	``commands.run`` and ``files.*`` take ``request_timeout`` only; handing them the full
	``ApiParams`` raises ``TypeError: unexpected keyword argument 'api_key'``. ``get_info``
	accepts ``ApiParams``, but the sandbox already carries the connection config, so the
	request timeout is enough.
	"""

[ananthsub]

Nit: this line runs to 116 characters while the rest of the page wraps at about 96. A reflow, which also drops the version from the old 300-second note (2.46 still uses 300):

Suggested change

	sandbox that is already gone as closed, and clears the local handle only after confirmed cleanup. Serialized handles
	carry the E2B sandbox ID for reconnection through another configured provider instance. E2B's
	public connect operation may extend a near-expiry sandbox to its SDK-default connection lifetime
	(300 seconds in E2B 2.36).
	sandbox that is already gone as closed, and clears the local handle only after confirmed
	cleanup. Serialized handles carry the E2B sandbox ID for reconnection through another
	configured provider instance. E2B's public connect operation may extend a near-expiry sandbox
	to its SDK-default connection lifetime (300 seconds).

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41537455431