Repository navigation
feat(nooa): stage a portable sandbox runtime - #3844
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
51fbbd4 to
4692a91
Compare
4692a91 to
e0ad444
Compare
e0ad444 to
b488222
Compare
b488222 to
c7e8559
Compare
c7e8559 to
ea5fc2b
Compare
abca166 to
54de732
Compare
54de732 to
0efed19
Compare
a269c5f to
c694841
Compare
c694841 to
1793dde
Compare
1793dde to
c0f7940
Compare
yaoyu-33
left a comment
There was a problem hiding this comment.
Rechecked at 317ae85. Musl now fails during the target probe before runtime downloads or installation. The exact-head NOOA suite passed: 217 tests, with 8 Linux-only installer tests skipped on macOS. No blocking issue found in this layer; full CI is still outstanding.
|
/ok to test 317ae85 |
Run one retry-safe NOOA activation per native session. Use the existing SingleAgentTurnEnvironmentServer for verification, rewards, episode deadlines and cleanup; return evidence and resource cookies at close. Remove the NOOA-specific environment, finish endpoint and legacy run route. Declare the standard environment and example routing explicitly. Instantiate each runner during agent-session seeding and keep ownership solely in session state, including reuse on idempotent seed retries. Return tool HTTP errors outside 5xx as model-visible feedback while retaining fatal handling for 5xx and transport failures. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>
Install pinned NOOA and current Gym source under /opt/nemo-gym-nooa using target-native Python. Reuse only a matching source fingerprint with successful imports, leaving task dependencies untouched. No issue needed: runtime layer of the NOOA sandbox RFC. Validation: four unit tests and actual aarch64 Linux DockerProvider install/import/cache-reuse smoke passed. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com> (cherry picked from commit e0ad444) Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com> NOOA-only rebuild of PR #3844 from b488222. NOOA-only scope: shared infrastructure remains identical to main. Signed-off-by: Nikhil Deshmudre <nikhil.deshmudre@gmail.com>
yaoyu-33
left a comment
There was a problem hiding this comment.
Rechecked the main rebase: this layer is unchanged, and inherited test compatibility is fixed. Exact-head NOOA suite: 218 passed, 8 Linux-only skips on macOS. No blocking regression found.
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38970095732 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38970607607 |
Stage the portable NOOA runtime for borrowed task sandboxes. This layer now builds on the standard single-agent-turn session integration.
This is part of the existing NOOA integration stack, based on #3848; no separate issue is needed for this requested architecture revision. Shared/base environment servers, shared agent contracts, generic sandbox code, and other harnesses remain fixed.
Patterns adopted from merged #3717 (single-agent-turn lifecycle), #3554/#3961 (Hermes sessions and sandbox cleanup), #3640 (Codex), #3626 (Pi), and open #4560 (borrowed DeepSWE/OpenCode sandbox): seed Resources, seed agent, activate, close agent and collect evidence/cookies, verify, close Resources.
Compatibility: in sandboxed NOOA, agent-owned background services stop before verification; the Resources-owned sandbox and persisted artifacts remain available. Tasks requiring a NOOA-started service during grading are unsupported by this lifecycle. The standard contract has an episode-wide deadline and no separate NOOA agent deadline; Terminal-Bench recipes retain verifier timeouts but do not reproduce canonical per-agent deadlines.
Current revision:
0efed19193e1bb3e906318a6c6486f64078188ee.Validation evidence:
Implementation pushed before writing or running new behavior tests or local checks, as requested. Existing tests received only mechanical naming/import updates before this push. Checks, exact-layer tests, live-model smoke rollouts, and current-head CI are pending for this revision. Earlier test and CI evidence applies only to earlier revisions and does not validate this change. No new benchmark scores are claimed.
Tests and validation
Current revision:
a55f1eddbf13448c30728dafd17ef0af8ac26dbc. Rebased the existing ten-PR stack onto main9ef5ed0f80a9f81c5d82af6588cd4cc956c4f5d1; assembled tip is196bd5e333abd6e8ee7a93f4c8805ae33f67a66a(#4357).Main compatibility adds only two test updates: #3846 gives the fake HTTP response its headers mapping for checkpoint-aware Environment seeding; #4088 checks serialized tool-output snapshots while retaining history, round-trip, and interpreter-object coverage. Both changes were independently reviewed. The rebase preserves upstream NOOA serialization and runtime configuration changes.
Validation on the exact rebased revisions:
pre-commit run --all-filespassed. DCO sign-offs, sequential layer ancestry, and the focused final diff were checked.Command from each revision's
responses_api_agents/nooa_agent:python -m pytest tests --import-mode=importlib -q -o addopts= -o log_cli=false --tb=short -rs. Exact early-layer tests use installed NOOA051472343211914222e24ce36d8752f4e86bbe43; the assembled tip uses coherent core/CLI/BenchAgent19caab169b018476ac433d040f6ae3f06aeff101. Commands and logs:/tmp/nooa-merge-main-candidate-20261016/final-tests-{3843,3846,3844,4357}/.Prior real-model evidence for the unchanged reviewer-fix behavior remains at
/tmp/nooa-review-fixes-20261016/live-final/, run on pre-rebase tip8185eb1f405ba476aab42b00a222f92dbec4364f: real Qwen output-token and iteration exhaustion each verified once; an actual iteration limit followed by injected cleanup failure remained fatal, skipped verification, and closed both sessions. This main sync adds test compatibility rather than new agent behavior, so that smoke was not rerun and is not presented as a smoke of the rebased tree. It used the integration-test Resources fixture and was not a new SWE-bench/OpenSandbox canary.GitHub CI is requested for the new heads. Optional cross-harness conformance failures also reproduce on main and are separate from the required
Nemo_CICD_Testgate. Existing branch names, bases, draft states, native groups, and description text outside this validation section are preserved.Required branch CI passed on this revision: 8,793 core tests, 921 sandbox tests, 95.07% coverage, and 226 NOOA tests (including Linux installer tests for #3844).
Signed-off-by: Nikhil Deshmudre nikhil.deshmudre@gmail.com