feat(nooa): restrict SWE task networking and prepare offline runtime

benchmarks/swebench/pro/nooa.yaml

@@ -5,18 +5,24 @@ config_paths:
   - responses_api_agents/nooa_agent/configs/nooa_agent.yaml
   - environment_servers/single_agent_turn_legacy/configs/single_agent_turn_legacy.yaml
 
+use_absolute_ip: true
+
 # This flat-row recipe binds its own environment instead of the generic NOOA environment.
-nooa_environment_server:
-  _delete_key: environment_servers
+nooa_environment_server: null
 
 swebench_pro_nooa_resources_server:
   _inherit_from: swebench_pro_resources_server
   resources_servers:
     swebench_pro:
       allowed_agents: [nooa_agent]
+      # Only the task sandbox is restricted; fresh grading sandboxes retain their settings.
+      task_network_policy:
+        model_server: ${swebench_pro_nooa_agent.responses_api_agents.nooa_agent.model_server}
       sandbox_config:
         # Outlive the episode, patch extraction, and final cleanup budgets.
         ttl_s: 25200
+        provider_options:
+          platform: {os: linux, arch: amd64}
       datasets:
         - name: swebench_pro
           type: benchmark
@@ -34,6 +40,9 @@ swebench_pro_nooa_agent:
       nooa:
         execution_mode: sandboxed
       max_policy_calls: null
+      # Built on the controller before seeding; task installation performs no downloads.
+      runtime_wheelhouse: ${oc.env:NOOA_SWE_RUNTIME_WHEELHOUSE,null}
+      runtime_wheelhouse_required: true
 
 swebench_pro_nooa:
   _inherit_from: single_agent_turn_legacy

fern/versions/latest/pages/agent-server/nooa-swe-networking.mdx

@@ -0,0 +1,121 @@
+---
+title: "Run NOOA SWE-bench Pro with restricted networking"
+description: "Prepare NOOA offline and restrict only the SWE-bench Pro task sandbox on OpenSandbox."
+position: 5
+---
+
+# Run NOOA SWE-bench Pro with restricted networking
+
+The NOOA SWE-bench Pro recipe enables a task-only OpenSandbox network policy.
+The SWE Resources server applies default-deny egress when it creates the task
+sandbox, before handing the sandbox to NOOA. NOOA borrows that Resources-owned
+sandbox. Fresh grading sandboxes continue using the benchmark's original
+network settings, images, test artifacts, and dependency preparation.
+Other SWE harnesses keep their existing network behavior.
+
+## Deployment requirements
+
+Use an OpenSandbox deployment with an egress sidecar and **`dns+nft` enforcement**.
+DNS-only filtering can be bypassed with a literal IP address. Provider API
+acceptance of a policy is not proof of enforcement; validate both blocked domain
+requests and blocked literal-IP requests against the actual deployment. See
+[OpenSandbox egress controls](/main/infrastructure/sandbox/opensandbox-best-practices#egress-controls-are-off-by-default).
+
+The allowlist contains the advertised **Gym model-server host**, which proxies
+inference to the configured upstream model. NOOA uses the Gym Responses API; the Gym model adapter
+may call a different upstream protocol, such as Chat Completions. It does not add package indexes,
+GitHub, or the upstream model service to the task allowlist. Advertise a concrete
+host reachable from OpenSandbox; loopback and wildcard bind addresses are rejected.
+The policy model reference is derived from the NOOA agent's `model_server`, so overriding that
+reference also changes the permitted endpoint.
+The recipe uses `use_absolute_ip: true`, but a controller behind NAT still needs a
+reachable advertised host and firewall rules for its selected server ports.
+
+<Warning>
+This policy allows **all ports and HTTP routes on the allowed host**. It does not
+limit access to a model API path, and it cannot isolate other services sharing
+that host or IP. Place the model endpoint accordingly and assess cohosted services.
+The current helper is a host allowlist, not a model-request authorization boundary.
+</Warning>
+
+Only the OpenSandbox network-policy provider is supported for restricted tasks.
+Unsupported providers, unreachable endpoint configuration, and provider policy
+creation errors fail explicitly; they never trigger an unrestricted retry.
+Choose the provider explicitly with a provider config overlay. The NOOA recipe does not load
+OpenSandbox credentials when another provider is selected. Existing unrestricted local/Docker configurations
+remain available only when the user explicitly sets `task_network_policy: null`; restricted configurations
+never disable their policy automatically. An intentionally online runtime also needs
+`runtime_wheelhouse_required: false` and `runtime_wheelhouse: null`. These choices permit internet access
+and are unsuitable for a restricted evaluation.
+
+Configure the OpenSandbox deployment outside the task: no task environment variable
+can turn a DNS-only deployment into `dns+nft` enforcement.
+
+## Prepare and run
+
+Use a controller with Python 3.13.14 or newer, package/GitHub access, and Docker
+capable of running a Linux amd64 build container. Install Gym and the NOOA agent
+requirements in your controller environment. Docker builds the runtime; the
+benchmark task and grading sandboxes run on OpenSandbox.
+
+Prebuilding is intentional: Docker, package indexes, and GitHub are available on the build controller,
+not necessarily on the agent host or inside restricted task sandboxes. Startup reports a setup error
+when the required bundle path is unset; it does not attempt an online task installation.
+
+From the repository root, prepare a fresh bundle and the shared SWE dataset:
+
+```bash
+export NOOA_SWE_RUNTIME_WHEELHOUSE=/absolute/path/to/new-owned-runtime-bundle
+python -m responses_api_agents.nooa_agent.offline_runtime \
+  --output "$NOOA_SWE_RUNTIME_WHEELHOUSE" --target x86_64/gnu
+python benchmarks/swebench/pro/prepare.py
+```
+
+The bundle contains portable Python, the pinned NOOA packages, the Gym runtime,
+and all dependency wheels. Its manifest binds the source, Python and wheel checksums,
+architecture, and builder glibc version. Task setup uses local wheels with
+`--no-index --no-deps` in a private prefix. It preserves the task image and its Python.
+Missing, incompatible, or corrupted bundles fail setup without an online fallback.
+Source or dependency changes require a fresh bundle directory. An existing empty output directory is
+accepted; a nonempty incompatible directory is preserved and rejected. Use an absolute bundle path or
+`~/...` (expanded by both preparation and agent configuration). Relative agent paths are rejected because
+the server working directory differs from the preparation shell. An existing bundle
+can be checked with the same command plus `--validate-only`.
+
+The recipe selects Linux amd64. An arm64 platform overlay requires a matching
+`aarch64/gnu` bundle and compatible task images. Task glibc must be at least the
+builder version; the default `manylinux2014` builder uses glibc 2.17, permitting older glibc task images
+such as Ubuntu 18.04. A custom newer builder still requires its newer glibc; musl targets are unsupported. Prepared bundles can be transferred
+between controllers with matching source and target.
+
+Supply `OPENSANDBOX_DOMAIN` and `OPENSANDBOX_API_KEY` privately, and configure
+`policy_base_url`, `policy_api_key`, and `policy_model_name` through Gym's normal
+`env.yaml` or CLI overrides. The Gym model-server host must be reachable from the
+sandbox. Set the NOOA agent's optional `context_window` to the model's known context
+size when needed. The planner and outgoing request use the same output cap: explicit Gym
+`max_output_tokens` wins; otherwise NOOA's `max_tokens`, `max_completion_tokens`, or `max_output_tokens`,
+including an `extra_body` cap, is sent as `max_output_tokens`. Conflicting NOOA caps are rejected.
+When no cap is supplied, no request cap is invented; a planning fallback remains planning-only.
+
+```bash
+gym env start --config benchmarks/swebench/pro/nooa.yaml \
+  --config nemo_gym/sandbox/providers/opensandbox/configs/opensandbox.yaml \
+  --config responses_api_models/vllm_model/configs/vllm_model.yaml
+
+gym eval run --no-serve --agent swebench_pro_nooa_agent \
+  --input benchmarks/swebench/data/swebench_pro_benchmark.jsonl \
+  --output /tmp/nooa-swebench-pro.jsonl --limit 1 --concurrency 1
+```
+
+## Validate the deployment
+
+Before increasing concurrency, check a fresh task sandbox installs the offline
+runtime and reaches the model server. Confirm external hostname and literal-IP
+requests are denied, grading completes in a separate sandbox with its original
+network settings, and both sandboxes are cleaned up. Unit tests and successful
+policy creation do not establish live network enforcement.
+
+The recipe uses the existing SWE verifier and retains its reward and failure
+semantics. NOOA execution, transport, worker, resource, and cleanup failures remain
+episode failures. Inspect the rollout and verifier diagnostics before interpreting
+benchmark results.

fern/versions/latest/pages/evaluation-tutorials/nooa-swe-bench-pro.mdx

@@ -33,10 +33,11 @@ NOOA-started service to remain live are not supported by this lifecycle.
 
 Cold runtime setup requires Linux x86_64 or aarch64 with glibc, `bash`, `tar`, `gzip`, `sha256sum`, `flock`,
 `tee`, `mv` with `-T` support, and writable `/opt/nemo-gym-nooa` and `/tmp`. The controller supplies Python;
-the sandbox needs outbound package and build-dependency access. Alpine/musl images are not supported.
-Model and Resources URLs must be reachable from the sandbox. A host-loopback URL is not automatically
-rewritten; configure `use_absolute_ip` / `default_host` for the deployment. No host virtual environment
-or provider credentials are copied into the runner.
+Alpine/musl images are not supported. Python packages normally require outbound access during cold installation.
+The NOOA SWE-bench Pro recipe instead requires a controller-prepared offline wheelhouse because its task sandbox
+allows only the Gym model-server host. Model and granted Resources URLs must be reachable from the sandbox;
+configure `use_absolute_ip` / `default_host` for the deployment. No host virtual environment or provider
+credentials are copied into the runner.
 
 ## Resource tools and observations
 
@@ -65,26 +66,35 @@ counter reward, model usage, resource calls, and trajectory. This example does n
 
 ## SWE-bench Pro
 
-The recipe reuses the existing Resources-owned task sandbox, patch extraction, and separate grading sandbox:
+The recipe reuses the existing Resources-owned task sandbox, patch extraction, and separate grading sandbox.
+It restricts only the task sandbox on OpenSandbox; grading retains its original networking. Prepare the
+runtime outside the task sandbox before startup:
 
 ```bash
+export NOOA_SWE_RUNTIME_WHEELHOUSE=/absolute/path/to/new-owned-runtime-bundle
+python -m responses_api_agents.nooa_agent.offline_runtime \
+  --output "$NOOA_SWE_RUNTIME_WHEELHOUSE" --target x86_64/gnu
 python benchmarks/swebench/pro/prepare.py
-gym env start --config benchmarks/swebench/pro/nooa.yaml --config model-provider.yaml
+gym env start --config benchmarks/swebench/pro/nooa.yaml --config model-provider.yaml \
+  --config nemo_gym/sandbox/providers/opensandbox/configs/opensandbox.yaml
 
 gym eval run --no-serve \
   --config benchmarks/swebench/pro/nooa.yaml --config model-provider.yaml \
+  --config nemo_gym/sandbox/providers/opensandbox/configs/opensandbox.yaml \
   --input benchmarks/swebench/data/swebench_pro_benchmark.jsonl \
   --output /tmp/nooa-swebench-pro.jsonl --limit 1 --concurrency 1 \
   +agent_name=swebench_pro_nooa_agent
 ```
 
-Use `model-provider.yaml` to supply the policy model and sandbox provider settings and credentials. Pass the
-same recipe and overlay to both commands: `--no-serve` does not inherit routing from the running head server.
-For Docker, configure Gym's published model
-and Resources addresses to be reachable from the container before running. Preparation preserves upstream grading
-metadata in the shared flat dataset. The existing `single_agent_turn_legacy` adapter separates that metadata
-from the Responses request before invoking NOOA. Swap the provider configuration for
-remote infrastructure without changing the agent recipe.
+Use `model-provider.yaml` to supply the policy model and sandbox provider settings, and provide
+`OPENSANDBOX_DOMAIN` and `OPENSANDBOX_API_KEY` privately. Pass the same recipe and overlay to both commands:
+`--no-serve` does not inherit routing from the running head server. The model-server host must be reachable
+from the sandbox; its reference is derived from the NOOA agent configuration. Restricted SWE tasks require
+an OpenSandbox deployment with `dns+nft` enforcement; unsupported providers fail explicitly. The policy
+allows all ports and routes on the model-server host, including any cohosted services. Preparation preserves upstream grading metadata in the shared flat dataset.
+The existing `single_agent_turn_legacy` adapter separates that metadata from the Responses request before
+invoking NOOA. See [NOOA SWE-bench Pro restricted networking](/main/agent-server/nooa-swe-networking) for offline
+build prerequisites and live isolation checks. Other SWE harnesses keep their existing network behavior.
 
 The overall deadline belongs to the Environment Server (`default_episode_timeout_seconds`). The standard
 task contract has no separate NOOA agent deadline. This recipe sets the task sandbox TTL to **25,200 seconds

resources_servers/swebench_pro/app.py

@@ -32,9 +32,9 @@
 from fastapi.exceptions import RequestValidationError
 from pydantic import BaseModel, ConfigDict, Field, ValidationError
 
-from nemo_gym.agents import lifecycle_observation
+from nemo_gym.agents import lifecycle_observation, restricted_network_policy, sandbox_server_url
 from nemo_gym.client import SESSION_ID_KEY
-from nemo_gym.config import get_global_config_dict
+from nemo_gym.config import ModelServerRef, get_global_config_dict
 from nemo_gym.core import (
     VERIFIER_ERROR,
     EpisodeId,
@@ -133,6 +133,18 @@ class PatchCaptureLostError(RuntimeError):
     """The agent sandbox did not report how ``git diff`` ended or did not return its output, so the patch is unknown."""
 
 
+class SWEBenchProTaskNetworkPolicy(BaseModel):
+    """Allow only the model server host for task egress, on every port and route.
+
+    Requires an OpenSandbox deployment enforcing network policies with dns+nft.
+    Grading sandboxes retain the original sandbox_config.provider_options.
+    """
+
+    model_config = ConfigDict(extra="forbid")
+
+    model_server: ModelServerRef
+
+
 class SWEBenchProResourcesServerConfig(BaseResourcesServerConfig):
     is_verifying_golden_patch: bool = False
     apply_anti_cheating: bool = True
@@ -162,6 +174,7 @@ class SWEBenchProResourcesServerConfig(BaseResourcesServerConfig):
     image_template: str | None = None  # Local SIF path with a checked provenance manifest.
     sandbox_provider: str
     sandbox_config: dict[str, Any]
+    task_network_policy: SWEBenchProTaskNetworkPolicy | None = None
 
 
 class SWEBenchProInstanceRequest(BaseModel):
@@ -333,10 +346,18 @@ async def _create_sandbox(
         self,
         body: SWEBenchProInstanceRequest,
         files: dict[str, str] | None = None,
+        *,
+        task: bool = False,
     ) -> AsyncSandbox:
         global_config_dict = get_global_config_dict()
         provider_config = resolve_provider_config(self.config.sandbox_provider, global_config_dict)
         provider_metadata = resolve_provider_metadata(self.config.sandbox_provider, global_config_dict)
+        provider_options = self.config.sandbox_config.get("provider_options", {})
+        if task and self.config.task_network_policy is not None:
+            model_url = sandbox_server_url(self.config.task_network_policy.model_server.name, require_reachable=True)
+            # Replace rather than extend any baseline allowlist. Grading keeps the original options.
+            provider_options = dict(provider_options or {})
+            provider_options["network_policy"] = restricted_network_policy(next(iter(provider_config)), [model_url])
         image_info = await asyncio.to_thread(self._image_info, body)
         spec = SandboxSpec(
             image=image_info["image"],
@@ -353,7 +374,7 @@ async def _create_sandbox(
             },
             resources=SandboxResources.from_mapping(self.config.sandbox_config.get("resources", {})),
             entrypoint=None,
-            provider_options=self.config.sandbox_config.get("provider_options", {}),
+            provider_options=provider_options,
         )
         sandbox = AsyncSandbox(provider_config)
         await sandbox.start(spec)
@@ -430,7 +451,7 @@ async def _typed_seed_response(self, session_id: str) -> ResourcesSeedSessionRes
 
     async def _seed_session(self, session_id: str, body: SWEBenchProInstanceRequest) -> SWEBenchProSeedSessionResponse:
         await self._stop_session_sandbox(session_id)
-        sandbox = await self._create_sandbox(body)
+        sandbox = await self._create_sandbox(body, task=True)
         # Keep ownership before initialization, including when cleanup itself fails.
         self._session_id_to_sandbox[session_id] = sandbox
         try:

resources_servers/swebench_pro/tests/test_app.py

@@ -133,7 +133,7 @@ def typed_seed_json(session_id: str, rollout_id: str = "rollout", task_data: dic
 def stub_task_sandboxes(monkeypatch: MonkeyPatch, server: SWEBenchProResourcesServer) -> AsyncMock:
     """Make typed seeds open fake task sandboxes; return the ``_create_sandbox`` mock."""
     create = AsyncMock(
-        side_effect=lambda body, files=None: SimpleNamespace(
+        side_effect=lambda body, files=None, *, task=False: SimpleNamespace(
             _handle=SimpleNamespace(sandbox_id="sandbox-id"),
             exec=AsyncMock(),
             serialize=AsyncMock(return_value={"sandbox_id": "sandbox-id"}),
@@ -674,8 +674,11 @@ async def stop_verifier() -> None:
     )
     verifier_sandbox = SimpleNamespace(stop=AsyncMock(side_effect=stop_verifier))
 
-    async def create_sandbox(body: SWEBenchProInstanceRequest, files: dict | None = None) -> SimpleNamespace:
-        if files is None:
+    async def create_sandbox(
+        body: SWEBenchProInstanceRequest, files: dict | None = None, *, task: bool = False
+    ) -> SimpleNamespace:
+        assert task is (files is None)
+        if task:
             events.append("create-task")
             return task_sandbox
         events.append("create-verifier")