Skip to content

feat(nooa): stage a portable sandbox runtime - #3844

Merged
init-nikhil merged 2 commits into
init-nikhil/nooa-native-toolsfrom
init-nikhil/nooa-sandbox-runtime
Oct 16, 2026
Merged

init-nikhil merged 2 commits into
init-nikhil/nooa-native-toolsfrom
init-nikhil/nooa-sandbox-runtime

Conversation

@init-nikhil

@init-nikhil init-nikhil commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Stage the portable NOOA runtime for borrowed task sandboxes. This layer now builds on the standard single-agent-turn session integration.

This is part of the existing NOOA integration stack, based on #3848; no separate issue is needed for this requested architecture revision. Shared/base environment servers, shared agent contracts, generic sandbox code, and other harnesses remain fixed.

Patterns adopted from merged #3717 (single-agent-turn lifecycle), #3554/#3961 (Hermes sessions and sandbox cleanup), #3640 (Codex), #3626 (Pi), and open #4560 (borrowed DeepSWE/OpenCode sandbox): seed Resources, seed agent, activate, close agent and collect evidence/cookies, verify, close Resources.

Compatibility: in sandboxed NOOA, agent-owned background services stop before verification; the Resources-owned sandbox and persisted artifacts remain available. Tasks requiring a NOOA-started service during grading are unsupported by this lifecycle. The standard contract has an episode-wide deadline and no separate NOOA agent deadline; Terminal-Bench recipes retain verifier timeouts but do not reproduce canonical per-agent deadlines.

Current revision: 0efed19193e1bb3e906318a6c6486f64078188ee.

Validation evidence:

Implementation pushed before writing or running new behavior tests or local checks, as requested. Existing tests received only mechanical naming/import updates before this push. Checks, exact-layer tests, live-model smoke rollouts, and current-head CI are pending for this revision. Earlier test and CI evidence applies only to earlier revisions and does not validate this change. No new benchmark scores are claimed.

Tests and validation

Current revision: a55f1eddbf13448c30728dafd17ef0af8ac26dbc. Rebased the existing ten-PR stack onto main 9ef5ed0f80a9f81c5d82af6588cd4cc956c4f5d1; assembled tip is 196bd5e333abd6e8ee7a93f4c8805ae33f67a66a (#4357).

Main compatibility adds only two test updates: #3846 gives the fake HTTP response its headers mapping for checkpoint-aware Environment seeding; #4088 checks serialized tool-output snapshots while retaining history, round-trip, and interpreter-object coverage. Both changes were independently reviewed. The rebase preserves upstream NOOA serialization and runtime configuration changes.

Validation on the exact rebased revisions:

Command from each revision's responses_api_agents/nooa_agent: python -m pytest tests --import-mode=importlib -q -o addopts= -o log_cli=false --tb=short -rs. Exact early-layer tests use installed NOOA 051472343211914222e24ce36d8752f4e86bbe43; the assembled tip uses coherent core/CLI/BenchAgent 19caab169b018476ac433d040f6ae3f06aeff101. Commands and logs: /tmp/nooa-merge-main-candidate-20261016/final-tests-{3843,3846,3844,4357}/.

Prior real-model evidence for the unchanged reviewer-fix behavior remains at /tmp/nooa-review-fixes-20261016/live-final/, run on pre-rebase tip 8185eb1f405ba476aab42b00a222f92dbec4364f: real Qwen output-token and iteration exhaustion each verified once; an actual iteration limit followed by injected cleanup failure remained fatal, skipped verification, and closed both sessions. This main sync adds test compatibility rather than new agent behavior, so that smoke was not rerun and is not presented as a smoke of the rebased tree. It used the integration-test Resources fixture and was not a new SWE-bench/OpenSandbox canary.

GitHub CI is requested for the new heads. Optional cross-harness conformance failures also reproduce on main and are separate from the required Nemo_CICD_Test gate. Existing branch names, bases, draft states, native groups, and description text outside this validation section are preserved.

Required branch CI passed on this revision: 8,793 core tests, 921 sandbox tests, 95.07% coverage, and 226 NOOA tests (including Linux installer tests for #3844).

Signed-off-by: Nikhil Deshmudre nikhil.deshmudre@gmail.com

@init-nikhil init-nikhil added area:agent Agent harnesses and Responses API agent behavior area:sandbox Sandbox providers, isolation, and sandbox runtime contracts labels Oct 1, 2026
@copy-pr-bot

copy-pr-bot Bot commented Oct 1, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@init-nikhil
init-nikhil added this pull request to stack #3852 October 1, 2026 06:39
@init-nikhil
init-nikhil force-pushed the init-nikhil/nooa-sandbox-runtime branch from 51fbbd4 to 4692a91 Compare October 8, 2026 02:23
@init-nikhil
init-nikhil force-pushed the init-nikhil/nooa-sandbox-runtime branch from 4692a91 to e0ad444 Compare October 8, 2026 21:01
@init-nikhil
init-nikhil force-pushed the init-nikhil/nooa-sandbox-runtime branch from e0ad444 to b488222 Compare October 9, 2026 21:58
@init-nikhil
init-nikhil removed this pull request from stack #3852 October 9, 2026 21:59
@init-nikhil
init-nikhil added this pull request to stack #4355 October 9, 2026 22:00
@init-nikhil
init-nikhil removed this pull request from stack #4355 October 10, 2026 05:25
@init-nikhil
init-nikhil force-pushed the init-nikhil/nooa-sandbox-runtime branch from b488222 to c7e8559 Compare October 10, 2026 05:25
@init-nikhil
init-nikhil added this pull request to stack #4367 October 10, 2026 05:25
@init-nikhil init-nikhil changed the title feat(nooa): [wip] stage a portable sandbox runtime feat(nooa): stage a portable sandbox runtime Oct 10, 2026
@init-nikhil
init-nikhil force-pushed the init-nikhil/nooa-sandbox-runtime branch from c7e8559 to ea5fc2b Compare October 10, 2026 05:29
@init-nikhil
init-nikhil marked this pull request as ready for review October 12, 2026 04:31
@nemo-automation-bot
nemo-automation-bot Bot requested a review from a team October 12, 2026 04:32
@init-nikhil
init-nikhil requested a review from yaoyu-33 October 12, 2026 04:37
@yaoyu-33 yaoyu-33 added complexity:low Localized change in one scope with a small, straightforward review surface feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer labels Oct 12, 2026
@init-nikhil
init-nikhil force-pushed the init-nikhil/nooa-sandbox-runtime branch from abca166 to 54de732 Compare October 15, 2026 23:20
@init-nikhil
init-nikhil force-pushed the init-nikhil/nooa-sandbox-runtime branch from 54de732 to 0efed19 Compare October 15, 2026 23:39
@init-nikhil
init-nikhil force-pushed the init-nikhil/nooa-sandbox-runtime branch 2 times, most recently from a269c5f to c694841 Compare October 16, 2026 00:11
@init-nikhil
init-nikhil removed this pull request from stack #4367 October 16, 2026 00:16
@init-nikhil
init-nikhil changed the base branch from init-nikhil/nooa-native-example to init-nikhil/nooa-native-sessions October 16, 2026 00:16
@init-nikhil
init-nikhil added this pull request to stack #4662 October 16, 2026 00:16
@init-nikhil
init-nikhil force-pushed the init-nikhil/nooa-sandbox-runtime branch from c694841 to 1793dde Compare October 16, 2026 02:23
@init-nikhil
init-nikhil force-pushed the init-nikhil/nooa-sandbox-runtime branch from 1793dde to c0f7940 Compare October 16, 2026 03:45
yaoyu-33
yaoyu-33 previously approved these changes Oct 16, 2026

@yaoyu-33 yaoyu-33 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked at 317ae85. Musl now fails during the target probe before runtime downloads or installation. The exact-head NOOA suite passed: 217 tests, with 8 Linux-only installer tests skipped on macOS. No blocking issue found in this layer; full CI is still outstanding.

@init-nikhil

Copy link
Copy Markdown
Member Author

/ok to test 317ae85

Run one retry-safe NOOA activation per native session. Use the existing
SingleAgentTurnEnvironmentServer for verification, rewards, episode
deadlines and cleanup; return evidence and resource cookies at close.
Remove the NOOA-specific environment, finish endpoint and legacy run
route. Declare the standard environment and example routing explicitly.

Instantiate each runner during agent-session seeding and keep ownership
solely in session state, including reuse on idempotent seed retries.

Return tool HTTP errors outside 5xx as model-visible feedback while
retaining fatal handling for 5xx and transport failures.

Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>
Install pinned NOOA and current Gym source under /opt/nemo-gym-nooa using target-native Python. Reuse only a matching source fingerprint with successful imports, leaving task dependencies untouched. No issue needed: runtime layer of the NOOA sandbox RFC. Validation: four unit tests and actual aarch64 Linux DockerProvider install/import/cache-reuse smoke passed.

Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>
(cherry picked from commit e0ad444)
Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>

NOOA-only rebuild of PR #3844 from b488222.

NOOA-only scope: shared infrastructure remains identical to main.

Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>

@yaoyu-33 yaoyu-33 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked the main rebase: this layer is unchanged, and inherited test compatibility is fixed. Exact-head NOOA suite: 218 passed, 8 Linux-only skips on macOS. No blocking regression found.

@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38970095732

@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38970607607

This branch was previously deployed

1 inactive deployment
public — a55f1edd Deployed Oct 16, 2026 by copy-pr-bot[bot] via release / finalize / notify #5694
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent harnesses and Responses API agent behavior complexity:low Localized change in one scope with a small, straightforward review surface feature New capabilities, enhancements, or enablement work ready-to-merge PR is approved, current, and waiting for final CI or merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants