[JeffPengCoder]

What does this PR do?

Separate the shared infrastructure changes inherited by #2554 and #3140 from OSWorld-specific behavior. The OSWorld/AgentENV companion #3592 and its dependency-metadata fix #3649 have merged. This branch now includes upstream main d96b33770 through normal merges; the original feature history is preserved.

Split (October 21). As suggested in review, the sandbox-provider changes moved to #4822 and the shared aiohttp lifecycle to #4823. This PR keeps only the component runtime policy, rollout identity and the vLLM transport-log context; none of it uses the moved code. Running OSWorld on AgentEnv needs #4822 as well, because main's E2B provider has no endpoint().

• Managed component runtimes: role-local Python/uv policy, parent-runtime constraints, environment identity recording, and resolved-source import/path protection.
• Rollout collection: forward a caller-supplied _ng_rollout_id unchanged; rows without one keep main's index-derived capture key. A caller's trajectory_identity names the trajectory record, and a payload-free dispatch summary is logged only when request debugging is enabled. No new execution-ID protocol.
• vLLM transport log: the opt-in NEMO_GYM_VLLM_TRANSPORT_LOG also records rollout correlation headers (rollout, group, source-group and sampling-event ids; rollout and attempt indices). The legacy compaction-session change was dropped, because refactor(agent): decouple simple-agent compaction from model serving #3368 replaces that session.
• Moved out: the sandbox providers and the E2B embedded-worker transport fallback are in fix(sandbox): reconcile Docker creates and resolve E2B service endpoints #4822; the owner-loop close and wildcard-host handling are in fix(core): close shared aiohttp clients on their owner loop #4823.

Synchronization with current main

The October 8 merge preserves upstream behavior rather than restoring older implementations:

• Configuration stays in the subprocess environment, not shell command arguments (fix(cli): pass config dict via process environment instead of command line (#3383) #3472). Resolved-source paths and the environment block are tested together.
• Servers use their explicit venv interpreter, and existing setup serialization/readiness behavior remains intact (fix(cli): launch servers with their venv interpreter and harden spin-up #3681, fix(cli): serialize shared virtual environment setup #3435).
• Bounded admission, dispatch budgets, non-retaining collection, ordering and cleanup remain intact (feat(rollout): bound driver task admission and support non-retaining collection #3497, feat(rollout): add a dispatch budget and an opt-in failure retry contract #3684, fix(rollout): two dispatch-budget correctness bugs #3853).
• Materialized episode requests keep main's episode identity rules. Legacy requests keep the caller's explicit capture ID; without one, main's index-derived capture key applies.
• HTTP retry controls, destination/pool telemetry and secret redaction remain intact (feat(core): add model retry controls and Gymnasium rollout options #3415, feat(core): expose aiohttp pool wait telemetry #3632, fix: stop leaking credentials in resolved configs, debug request logs and the vLLM transport log #3679).

Scope and compatibility

There is no main-relative diff under benchmarks/osworld, resources_servers/osworld, or responses_api_agents/osworld_agent. Their current code is inherited from main. No task definitions, prompts, history-policy configuration, rewards, sampling parameters or training recipes are changed by this synchronization.

#3649 already fixed OSWorld's Python 3.13/NumPy dependency conflict at the package-metadata source. This PR does not restore NumPy overrides. Generic role-local uv.toml handling remains relevant for CPU Torch selection and isolation from enclosing training workspaces; parent-sensitive packages keep the parent runtime's authority.

Validation

October 21: review fixes

Commits 513300d77, dc9943eda, 2dcb8d9a2 and 39dfe1c57 address the review, and 2d8d8e521 moves the provider and HTTP-lifecycle files to #4822 and #4823. They were tested on macOS ARM64 with Python 3.13.14 and the same dependencies as below (aiohttp 3.14.4, yarl 1.25.1, FastAPI 0.135.2, real E2B SDK 2.46.0):

• The core unit suite plus the vLLM, simple-agent, compaction-agent and OSWorld tests on head 2d8d8e521: 10,537 passed. No test that passes on main d96b33770 fails here; the remaining failures are identical on main and specific to this macOS machine. uv lock --check --offline passed.
• New or changed tests cover:
  • both component-root layouts; the wheel-install case fails without the guard;
  • index-derived capture keys for legacy rows, including a resumed attempt;
  • dispatch logging only with request debugging enabled;
  • the transport log recording the post-override sampling values.
• pre-commit run on the changed files: passed.

Together with #4822 and #4823, which carry the measured head's provider and HTTP-lifecycle changes unchanged, these fixes do not change the OSWorld path measured below:

• In a source checkout, the component-root guard keeps the previous behavior.
• Rollout collection no longer adds a _ng_rollout_id field and logs dispatches only on request.
• The vLLM change removes a print.
• OSWorld does not use the legacy compaction session.

October 20 synchronization: OSWorld score reproduction

The synchronized head 6055f9ea3 (main d96b33770 merged; #3139 is in main as 415ec2989) reproduces the OSWorld score: no degradation.

Setup. One run on the OSWorld 361-task set (the 369-task suite without the 8 Google Drive tasks), with the same settings as the #3139 score reproduction. Model: Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 (HF revision 24e67ea0), served by vLLM 0.23.0 with TP8 on 8×H100. Settings: fixed-3 image history (the benchmark default), 100 steps, temperature 0.6, top_p 0.95, 16,384 max output tokens. Desktops ran on an AgentEnv (E2B-compatible) deployment with task concurrency 128. Of the score-defining inputs, only the Gym source changed. That head also contained the provider and HTTP-lifecycle changes now in #4822 and #4823.

Run	Gym source	Raw SR	Binary SR
this PR, synchronized	6055f9ea3	48.13%	166/361 (45.98%)
#3139 reproduction, PR arm	main d4c0f86c7 + #3591 + #3139	46.62%	159/361 (44.04%)
#3139 reproduction, control arm	main d4c0f86c7 + #3591	44.61%	154/361 (42.66%)

• Difference. Against the PR arm, Δ raw is +1.51 pp, with a paired 95% bootstrap interval of [−2.56, +5.58] pp. For binary SR, 31 tasks succeeded only in this run and 24 only in the PR arm (McNemar exact p = 0.42). Against the control arm, Δ raw is +3.52 pp, interval [−0.78, +7.89] pp; binary 38 vs. 26 (p = 0.17). Both intervals contain 0. Repeated identical runs have differed by 1.2–2.8 pp.
• Coverage. All 361 tasks were scored: none masked, none missing. One task failed in environment setup, before any model call, on a transient download of a task setup file. It was re-run alone against the same model server and scored 1.0. The other 360 rows were not re-run.
• Same policy and contract. All 361 tasks have the same history_policy_id (osworld-history-policy-d171b7730c73e7c384ab2b25) and agent_contract_id (osworld-agent-contract-cb1698f34a75d3dbf5fd5063) as both feat(osworld): add sink-window history policy #3139 arms.
• Terminations. 260 agent_done, 89 at the 100-step limit, 5 agent_fail and 7 model_response_unparseable. Infeasible tasks score only on an explicit model FAIL: 3 of the 29 scored, all through agent_fail; the 13 that reached the step limit scored 0.
• Not exercised here. No context overflow occurred, so the shrink path was not exercised. The opt-in sink-window policy was not run.

Deployment notes, with no score effect:

• Current main makes Ray optional (feat(core): run Gym without Ray installed #4300), and the OSWorld agent declares ray_enabled = True, so the root environment needs the ray extra.
• feat(core): run Gym without Ray installed #4300 also passes an explicit -c when it builds server venvs, and uv then ignores UV_CONSTRAINT (observed with uv 0.11.32). A deployment that pins e2b in a server venv through UV_CONSTRAINT must re-apply that pin after the venv is built.

October 9 synchronization

Merge commit 4547ad10ae5f1c0c78d8122902fb33042babcea7 includes main 8908cb24d. The only textual merge conflict was uv.lock: the resolution preserves both e2b>=2.46.0,<3.0.0 and upstream's fastapi!=0.142.*. No package versions differ from upstream main's lockfile, and no changes from #4128 or #3368 were merged.

• 2,804 passed, zero skipped, one platform-specific test deselected on macOS ARM64 / Python 3.13.14.
• Final regression dependencies: aiohttp 3.14.4, yarl 1.25.1, FastAPI 0.135.2, and real E2B SDK 2.46.0. HTTP dependencies were resolved into a temporary test-only directory; existing training environments were not modified.
• The selection below was extended with test_cli_endpoint_readiness.py, test_ray_opt_out.py, test_responses_converter.py, test_responses_api_model_streaming.py, and tests/unit_tests/h2_ping_sidecar.
• uv lock --offline, uv lock --check --offline, source/lock constraint and version-boundary checks, pre-commit run --all-files, and the main-relative git diff --check: passed.
• The deselected procfs test is the same previously reproduced macOS/main failure described below. Full repository coverage, the eight-shard server suite, and live GPU/VM rollouts were not run for this synchronization.

October 8 validation

Against the final synchronized code, on macOS ARM64 / Python 3.13.14:

• 2,235 passed, zero skipped; one platform-specific test deselected in the final run. The full selection was also run before the final non-overlapping upstream update: its sole failure, test_memory_profiler_uses_spawned_server_metadata, assumes Linux procfs. It fails identically on a separate, unmodified checkout of main 64a734ed7; no platform workaround was added to this PR.
• The focused suites cover Core/CLI, generated installer subprocesses, Docker/OpenSandbox/E2B, rollout collection/correlation, episode/session APIs, vLLM, compaction, and OSWorld agent/resources/benchmark interfaces.
• Optional E2B 2.46.0 was supplied in an isolated temporary dependency directory, exercising the real SDK API/call-shape and shared local HTTP tests without live sandbox credentials. Existing training environments were not modified.
• pre-commit run --all-files, uv lock --check --offline, and the main-relative git diff --check: passed.

Test selection, with the development and optional E2B dependencies available:

NEMO_GYM_EXTRA_ROOTS="$PWD" PYTHONPATH="$PWD" RAY_TMPDIR=/tmp python -m pytest \
  tests/unit_tests/test_{base_resources_server,cli,cli_setup_command,cli_server_launch,global_config,docker_provider,e2b_endpoint,e2b_provider,e2b_sdk_transport_fallback,opensandbox_provider,opensandbox_http_transport,rollout_collection,rollout_correlation,server_utils,episode_types,environment_server,agent_sessions,resources_sessions,sandbox_access,tool_access}.py \
  responses_api_models/vllm_model/tests \
  responses_api_agents/simple_agent_with_compaction/tests/test_compaction.py \
  responses_api_agents/osworld_agent/tests resources_servers/osworld/tests benchmarks/osworld/tests \
  --import-mode=importlib -q -o addopts='' -o log_cli=false \
  -k 'not test_memory_profiler_uses_spawned_server_metadata'

This is full-scope under the repository CI classifier. The complete repository coverage gate and eight-shard server suite were not run locally and still require repository CI.

Rollout validation scope

The October 8 and 9 synchronizations were validated without GPU rollouts, live sandboxes/VMs or benchmarks. The October 20 synchronization was validated by the 361-task OSWorld score reproduction above, and the October 21 review fixes by unit tests; those fixes do not change the path that reproduction measured. No GRPO or training run was launched, so none of these checks claims full training completion. No internal credentials, deployment details or private run links are included.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[Glorf]

/ok to test 47f3fcf

[aroshanghias-nvd]

This overlaps with our context compaction refactor in #3368. We remove ContextCompactionSession, so the final_policy_decision fix won’t carry over directly; the replacement checks guards before generation and acknowledges history after accepting the response. The rest of your infrastructure changes look largely independent. Rather than stack the whole PR, let’s coordinate merge order and preserve both sets of behavior in rollout collection and the model proxy.

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/39656066230

[yaoyu-33]

@JeffPengCoder I pushed c004893 to this branch. It addresses the remaining runtime-policy points from review rather than leaving them as follow-ups. It is a fast-forward on top of your e3ea83a92, so nothing of yours was rewritten. Feel free to adjust or revert any part of it.

What changed (nemo_gym/cli/setup_command.py, its tests, the vLLM runtime test, and reference/configuration.mdx):

1. The server Python stays on the parent's patch release.
   • Problem: a component .python-version range such as >=3.13.14,<3.14 replaced the parent-exact python_version, which global_config.py pins because Ray checks Python at patch level. That let uv pick a different 3.13.x for osworld_agent, which is ray_enabled.
   • New rule: when the request admits the parent's Python, the venv uses the parent's exact version. Only a request that excludes it is passed to uv as written.
   • The request may be a range, 3.13, or cpython@3.13. Variant, pre-release and other-implementation requests (3.13+freethreaded, 3.13.14+debug, pypy@3.10) still go to uv unchanged.
   • As in uv, the first line that is not a comment is used.
2. Only the Ray pin is an override.
   • Before: head_server_deps (the parent's openai, telemetry and plugin packages) were written to an overrides file, which silently replaced a server's own conflicting pin. A conflict used to fail the install.
   • Now: only head_server_constraints (the Ray pin) are passed as --overrides, through process substitution like the existing -c. Your fix for stale transitive Ray pins is unchanged. Parent dependencies are plain requirements again, so conflicts fail loudly.
3. The environment-identity file is removed.
   • Nothing read .nemo-gym-environment-identity, so it could not detect a stale venv.
   • The tests that covered it used skip_venv_if_present=False, so they passed whatever it contained.
   • If you want stale-venv detection, it would need to be wired into the --skip-if-ready path. That seemed better as its own change.
4. Docs. Per-server .python-version, uv.toml, the parent-pin rules and NEMO_GYM_ALLOWED_COMPONENT_ROOTS are documented under "Server Virtual Environments".
5. The vLLM runtime test is replaced. The marker-file test asserted that files that never existed were absent. The new test checks through setup_env_command that the model venv is built with the parent's Python, and keeps the <3.14 check.

Validation on macOS ARM64, Python 3.13.14, uv 0.8.18:

• Unit tests:
  • test_cli_setup_command.py, test_cli.py, test_global_config.py, test_rollout_collection.py and vllm_model/tests: 1,520 passed.
  • The only failure is test_memory_profiler_uses_spawned_server_metadata, the macOS procfs failure that also happens on main.
  • With the old setup_command.py, 21 of the setup-command tests fail.
• pre-commit run on the changed files: passed.
• Real uv runs of the generated commands:
  • Parent 3.13.14 with >=3.13.14,<3.14 builds the venv on 3.13.14.
  • Ray is absent when the server does not need it.
  • A server that requires Ray gets 2.58.0.
  • A server pinning ray==2.54.0 is overridden to 2.58.0.
  • A server pinning packaging==25.0 against a parent packaging==24.2 fails the install.

[yaoyu-33]

Re-approving at c004893. My earlier approval was dismissed when I pushed the runtime-policy follow-up; that commit and its validation are described in the comment above. The OSWorld-only x-nemo-gym-log-* follow-up in vllm_model still stands as a separate change.

[yaoyu-33]

/ok to test c004893

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-3591-c004893a9da8.docs.buildwithfern.com/nemo/gym

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/39691150761