feat(sandbox): support AgentENV auto-pause and resume

fern/versions/latest/pages/infrastructure/sandbox/e2b.mdx

@@ -43,6 +43,7 @@ uses the operator's authentication settings. Set `E2B_API_URL` and
 Gateway-specific create parameters go in `create.custom_extension_params` and are sent as the
 API's `customExtensionParams`. The E2B SDK cannot send that field, so setting it makes the
 provider create each sandbox with one `POST {api_url}/sandboxes` and then attach with `connect`.
+That request sends the same timeout lifecycle (`create.auto_resume`) as the SDK create.
 
 ## Provider Config
 
@@ -63,6 +64,7 @@ sandbox:
       template: null
       template_map: {}
       timeout_s: 3600.0
+      auto_resume: false
       secure: true
       allow_internet_access: true
       network: null
@@ -147,8 +149,11 @@ suitable template prepared through E2B tooling.
 
 ## Timeouts and Reconnection
 
-`SandboxSpec.ttl_s` controls the remote sandbox lifetime. Treat it as a cleanup backstop and
-still call `stop()` or use a context manager so the sandbox is killed as soon as work finishes.
+`SandboxSpec.ttl_s` controls the remote sandbox lifetime. With the default
+`create.auto_resume: false`, the sandbox is deleted when it elapses: treat it as a cleanup
+backstop and still call `stop()` or use a context manager so the sandbox is killed as soon as
+work finishes. With `auto_resume: true` it pauses instead; see
+[AgentENV auto-pause and auto-resume](#agentenv-auto-pause-and-auto-resume).
 
 `SandboxSpec.ready_timeout_s` overrides the E2B create request timeout. The SDK retains an
 explicit create timeout on the returned sandbox connection; the shipped
@@ -169,8 +174,8 @@ rather than starting a new one.
 
 Create is not automatically retried because an ambiguous failure could allocate two billable
 sandboxes. Rate-limit responses are returned without automatic retries so callers can choose
-their backoff window. Close retries transient kill failures, treats an expired sandbox as
-already closed, and clears the local handle only after confirmed cleanup. Serialized handles
+their backoff window. Close retries transient kill failures, deletes paused sandboxes, treats a
+sandbox that is already gone as closed, and clears the local handle only after confirmed cleanup. Serialized handles
 carry the E2B sandbox ID for reconnection through another configured provider instance. E2B's
 public connect operation may extend a near-expiry sandbox to its SDK-default connection lifetime
 (300 seconds in E2B 2.36).
@@ -214,3 +219,34 @@ E2B's set-once integration hook. Because E2B 2.x does not expose transport injec
 APIs, NeMo Gym replaces its module-level HTTP transport factories with an aiohttp adapter backed
 by Gym's shared client session. The adapter accepts HTTP and HTTPS proxies; SOCKS proxies are
 rejected explicitly. E2B's ConnectRPC command streams keep their SDK-owned pyqwest transport.
+
+## AgentENV auto-pause and auto-resume
+
+`e2b.create.auto_resume` selects what happens when a sandbox reaches its timeout:
+
+| Value | Timeout action |
+| --- | --- |
+| `false` (default) | Delete the sandbox. |
+| `true` | Pause the VM, keeping its files and processes; the next sandbox request resumes it (AgentENV). |
+| `null` | Send no lifecycle and leave the action to the SDK and server. |
+
+Avoid `null` on AgentENV. E2B 2.36–2.41 send kill-on-timeout when no lifecycle is given, but
+2.42 and later (including the locked 2.46) omit the fields. AgentENV then pauses the sandbox
+without auto-resume and never evicts it, so a crashed or cancelled run leaks paused VMs. The
+lifecycle settings require a deployment that supports the E2B lifecycle API.
+
+`create.timeout_s` controls the initial timeout, and `SandboxSpec.ttl_s` overrides it per
+sandbox. It is a lifetime deadline, not an idle-traffic timer: choose a value appropriate for
+long-running commands. With `auto_resume: true`, note these AgentENV behaviors (see the
+[proxy contract](https://github.com/kvcache-ai/AgentENV/blob/f9a39636caa3d234bbb96d2406685d7a6d3dabc4/docs/src/internals/proxy-design.md#paused-sandbox-auto-resume)):
+
+- After an automatic resume, the sandbox gets at least the operator's minimum timeout
+  (default 300 seconds), counted from the resume. A shorter initial timeout does not change it.
+- Requests that arrive while the VM is pausing or resuming fail immediately with HTTP 410, so a
+  call at the timeout boundary can fail even with auto-resume enabled.
+- Attaching to a sandbox wakes it: `connect()` sends the SDK's 300-second default timeout,
+  which replaces the stored timeout, so later automatic resumes also get only 300 seconds.
+
+Status checks use the control plane and report `paused` without waking the VM. Explicit close
+still deletes the sandbox, including paused sandboxes. Auto-pause preserves storage; it does
+not replace cleanup after a completed or crashed run.

nemo_gym/sandbox/providers/e2b/README.md

@@ -73,7 +73,7 @@ provisioning the remaining images.
 | Section | Important fields |
 | --- | --- |
 | `connection` | SDK credentials, hosted or gateway URLs, headers, proxy, and control-plane `request_timeout_s`. |
-| `create` | Default template/mapping, sandbox `timeout_s`, `secure`, `allow_internet_access`, `network` (`allow_out`/`deny_out`), `custom_extension_params` (AgentEnv `customExtensionParams`; creates through `POST /sandboxes`), and `strict_resources`. |
+| `create` | Default template/mapping, sandbox `timeout_s`, timeout action `auto_resume`, `secure`, `allow_internet_access`, `network` (`allow_out`/`deny_out`), `custom_extension_params` (AgentEnv `customExtensionParams`; creates through `POST /sandboxes`), and `strict_resources`. |
 | `exec` | Fallback command timeout, default user, stream-open request timeout, background execution, and reconnect attempts. |
 | `operations` | Retry count and exponential-backoff delays for safe, retryable operations. |
 
@@ -117,10 +117,12 @@ suitable E2B template prepared through E2B tooling.
   first, billable sandbox running while a retry creates another one.
 - Rate-limit responses are returned without automatic retries so callers can choose an
   appropriate backoff window.
-- Close retries transient kill failures and treats an already-expired sandbox as closed. The
-  local handle is cleared only after E2B confirms the sandbox is killed or already absent.
-- The configured TTL is a remote cleanup backstop, not a replacement for `stop()` or a context
-  manager.
+- Close retries transient kill failures and treats an already-deleted sandbox as closed. It
+  also deletes paused sandboxes. The local handle is cleared only after E2B confirms the
+  sandbox is killed or already absent.
+- With the default `create.auto_resume: false`, the TTL deletes the sandbox and acts as a
+  remote cleanup backstop; it is not a replacement for `stop()` or a context manager. With
+  `auto_resume: true` the TTL pauses the sandbox instead, so it keeps its storage until closed.
 - Background commands can reconnect by process ID after a stream interruption. Output emitted
   before reconnection is not replayed, and a process that exits during the gap cannot be
   recovered. Reconnects share the original command wait budget rather than restarting it.

nemo_gym/sandbox/providers/e2b/configs/e2b.yaml

@@ -64,6 +64,9 @@ sandbox:
       template_map: {}
       # Default sandbox lifetime. SandboxSpec.ttl_s overrides this per sandbox.
       timeout_s: 3600.0
+      # false deletes at timeout; true pauses and resumes on the next request (AgentENV).
+      # null sends no lifecycle, which can leave timed-out sandboxes paused on AgentENV.
+      auto_resume: false
       # Secure the sandbox envd service with an access token.
       secure: true
       # Permit outbound internet access. Set false for offline/untrusted workloads.

nemo_gym/sandbox/providers/e2b/provider.py

@@ -184,9 +184,12 @@ class E2BCreateConfig:
     # image references are not themselves valid aliases (registry refs contain
     # '/' and ':', which E2B rejects).
     template_map: dict[str, str] = field(default_factory=dict)
-    # Sandbox lifetime in seconds; E2B kills the sandbox when it elapses.
+    # Sandbox timeout in seconds; the lifecycle policy selects pause or kill.
     # ``SandboxSpec.ttl_s`` overrides it per sandbox.
     timeout_s: float | None = 3600.0
+    # True pauses/resumes; False kills at timeout; None sends no lifecycle. E2B 2.42+ then leaves
+    # the choice to the server, and AgentENV pauses without resume and never evicts the sandbox.
+    auto_resume: bool | None = field(default=False, kw_only=True)
     allow_internet_access: bool = True
     secure: bool = True
     # Outbound traffic rules, passed as the SDK's ``network`` option:
@@ -225,6 +228,8 @@ def __post_init__(self) -> None:
                     raise ValueError(f"create.network.{key} must be a list of strings")
         if self.custom_extension_params is not None and not isinstance(self.custom_extension_params, Mapping):
             raise TypeError("create.custom_extension_params must be a mapping")
+        if self.auto_resume is not None and not isinstance(self.auto_resume, bool):
+            raise ValueError("create.auto_resume must be a boolean or None")
 
 
 @dataclass(frozen=True)
@@ -304,13 +309,7 @@ def _api_params(self) -> dict[str, Any]:
         return params
 
     def _request_params(self) -> dict[str, Any]:
-        """Per-request options for calls on an existing sandbox object.
-
-        ``commands.run``, ``files.*`` and ``is_running`` take ``request_timeout``
-        only -- the sandbox already carries the connection config, and handing
-        them the full ``ApiParams`` raises ``TypeError: unexpected keyword
-        argument 'api_key'``.
-        """
+        """Return the request timeout; data-plane methods reject connection parameters."""
         if self._connection.request_timeout_s is None:
             return {}
         return {"request_timeout": self._connection.request_timeout_s}
@@ -460,6 +459,11 @@ async def create(self, spec: SandboxSpec) -> SandboxHandle:
             "secure": self._create.secure,
             **self._api_params(),
         }
+        if self._create.auto_resume is not None:
+            kwargs["lifecycle"] = {
+                "on_timeout": "pause" if self._create.auto_resume else "kill",
+                "auto_resume": self._create.auto_resume,
+            }
         if spec.ready_timeout_s is not None:
             # E2B retains this on the returned sandbox connection. Subsequent
             # provider calls explicitly reapply connection.request_timeout_s
@@ -505,6 +509,10 @@ async def _create_with_extension_params(self, e2b: Any, kwargs: dict[str, Any])
         }
         if "network" in kwargs:
             body["network"] = {_NETWORK_API_KEYS[key]: rules for key, rules in kwargs["network"].items()}
+        if "lifecycle" in kwargs:
+            # The SDK's wire form of the lifecycle, so both create paths time out alike.
+            body["autoPause"] = kwargs["lifecycle"]["auto_resume"]
+            body["autoResume"] = {"enabled": kwargs["lifecycle"]["auto_resume"]}
         client = e2b.api.client_async.get_api_client(config).get_async_httpx_client()
         response = await client.post("/sandboxes", json=body)
         if response.status_code >= 300:
@@ -608,12 +616,15 @@ async def status(self, handle: SandboxHandle) -> SandboxStatus:
             if isinstance(exc, type)
         )
         try:
-            running = await sandbox.is_running(**self._request_params())
+            # Control-plane status avoids waking paused sandboxes with a health check.
+            info = await sandbox.get_info(**self._request_params())
         except not_found:
             return SandboxStatus.STOPPED
         except Exception:  # noqa: BLE001 - status must not raise for transient issues
             return SandboxStatus.UNKNOWN
-        return SandboxStatus.RUNNING if running else SandboxStatus.STOPPED
+        return {"running": SandboxStatus.RUNNING, "paused": SandboxStatus.PAUSED}.get(
+            info.state, SandboxStatus.UNKNOWN
+        )
 
     async def close(self, handle: SandboxHandle) -> None:
         e2b = _require_e2b_sdk()