Repository navigation
fix(osworld): resolve Python 3.13 NumPy metadata conflict - #3649
Merged
cmunley1 merged 1 commit intoSep 23, 2026
Merged
Conversation
Consume the OSWorld fork's Python-conditional NumPy/OpenCV requirements in both managed roles, remove redundant NumPy overrides, and keep source documentation and dependency-policy tests aligned. This fixes the server-suite resolver regression from NVIDIA-NeMo#3592 without depending on shared installer changes in NVIDIA-NeMo#3591. Signed-off-by: Jeff Peng <jepeng@nvidia.com>
JeffPengCoder
added
bug
cmunley1
approved these changes
Sep 23, 2026
Contributor
|
/ok to test 0274e3e |
JeffPengCoder
added a commit
to JeffPengCoder/Gym
that referenced
this pull request
Sep 23, 2026
Signed-off-by: Jeff Peng <jepeng@nvidia.com>
This was referenced Sep 23, 2026
pull Bot
pushed a commit
to Stars1233/Gym
that referenced
this pull request
Sep 23, 2026
) This follows merged NVIDIA-NeMo#2814, which established group, attempt, and member identities. It addresses NVIDIA-NeMo#3181's immediate timeout and failure-cleanup requirements. - **While collecting answers:** enforce a finite deadline. If a member never arrives, fail the group and release waiting requests without publishing partial rewards. - **Once all answers are available:** retry transient judge HTTP failures, including interrupted response bodies, within a bounded budget using the same answers. - **If judging ultimately fails:** preserve each generated answer and return an explicitly masked failure. The zero reward is a placeholder, excluded from Gym evaluation metrics by default. - **On failure, cancellation, or shutdown:** release waiters and clean up pending work. ### Recovery and compatibility A judge failure returns HTTP 200 with `mask_sample: true`, `failure_kind: judge_failed`, a bounded `failure_reason`, and the existing failure-sidecar tags. The shared judge failsafe also preserves `instance_config.mask_sample: true` for older RL consumers. This shared response fix applies to every environment using `judge_failsafe`. The existing evaluation opt-in `count_failure_classes_as_zero: [judge_failed]` still includes selected failures as zeros. Only the metric-input copy is unmasked; saved failure records and training masks remain unchanged. Online and offline aggregation apply the same policy, including when every rollout fails. With the opt-in, an all-`judge_failed` run reports `mean/reward: 0.0`; without eligible counted failures or successful results, collection still raises. The main rollout JSONL never receives synthetic successes. **A masked HTTP 200 result does not automatically make RL retry the group.** It is a completed, unusable result for the current RL caller. If the caller chooses replacement: - **Explicit-ID groups:** advance the shared group-attempt number for every member and submit a complete replacement group. - **Legacy groups without an ID:** failed groups retain a failure record. Reusing their task/prompt key returns failure with migration guidance; recovery requires a fresh explicit group ID for the complete group. This prevents delayed old members from joining a replacement. Successful legacy groups still allow sequential new evaluations. The judge-request default is now **1,800 seconds**, matching the overall evaluation budget. This allows substantially more room for long generations and queueing than the previous 300-second default; deployments must still size both limits for their workload. **Required RL coordination:** before NeMo RL adopts this Gym revision, coordinate with [RL NVIDIA-NeMo#4160](NVIDIA-NeMo/RL#4160) / [NVIDIA-NeMo#4061](NVIDIA-NeMo/RL#4061). Reading the mask alone is insufficient: TransferQueue must preserve it in the training loss mask, and unavailable judge rewards must be excluded from group statistics. NVIDIA-NeMo#4160 includes these fixes with `masked_reward_policy: exclude` as the default. Top-level field ingestion remains separate migration work, so the nested compatibility field remains. Before updating RL's Gym revision, validate a real failed Gym response through RL's loss mask and group statistics. Gym can merge first if RL keeps its existing Gym revision until those fixes are available. GenRM-specific usage and recovery documentation is now in the [resources-server README](https://github.com/NVIDIA-NeMo/Gym/blob/yuhez/3181-genrm-cohort-lifecycle/resources_servers/genrm_compare/README.md#genrm-comparison-groups), rather than a new Fern site page. Closes NVIDIA-NeMo#3181. <details> <summary>Validation</summary> Current code: `6a7f30b707aee62c46478d757005f3fcc9fabe0d`, incorporating main at `d4c0f86c7`. The latest fix is `059adb409`. - **5,110 core, GenRM, and OSWorld dependency-policy tests passed** on Python 3.13.14: `python -m pytest tests/unit_tests resources_servers/genrm_compare/tests responses_api_agents/osworld_agent/tests/test_dependency_policy.py -m 'not sandbox' -q --tb=short`. There were 103 skips, 673 sandbox tests deselected, and 184 passing subtests. Sandbox tests were not run locally. - The all-failure regression checks one and four masked judge failures, zero counting enabled/disabled, an unrelated counted class, and deferred aggregation. Four cases fail on the preceding code and pass with this fix; default and wrong-class error behavior remains intact. Existing mixed-success tests remain covered. - Actual TCP collector → SimpleAgent → GenRM tests check both collection timeout and judge outage, with zero counting on/off. Online and offline metrics match, the failure sidecar remains byte-for-byte unchanged, and the main/merged rollout JSONL stays empty. - Final all-files pre-commit, whitespace checks, and DCO sign-off passed. [Exact-head CI](https://github.com/NVIDIA-NeMo/Gym/actions/runs/35867697700) passed, including core and all eight server shards. The preceding run passed core and seven server shards but failed OSWorld dependency resolution; merged NVIDIA-NeMo#3649 fixes that NumPy metadata conflict and is included in this head. [Previous-head CI](https://github.com/NVIDIA-NeMo/Gym/actions/runs/35681811360) passed on `be7e6353d`, including core and all eight server shards. - No new model run for the latest collector aggregation fix: it changes metric eligibility, not policy generation or judging. The earlier real-model evidence below is from `be7e6353d`. - **Real-model functional validation passed:** Slurm job **19084624**, exit 0 in **14m58s**, on `be7e6353d`. Gym's native CLI/HTTP path used Qwen3-0.6B policy and Qwen3-Nemotron-235B-A22B-GenRM-2603 on eight H100s. Eight policy generations and four completed real judge calls were captured. - The run preserved four masked failure answers during an injected judge outage, then scored a caller-coordinated complete replacement attempt. Four HTTP 503s and four interrupted HTTP response bodies recovered within the retry budget without extra policy generation. A second collector resume made no further model calls. Failed legacy groups rejected delayed members, explicit zero-count aggregation preserved saved masks, and rewards exactly matched the captured judgments. Saved artifacts were independently inspected. - The model run enabled `CUDA_LAUNCH_BLOCKING=1` for diagnostics after an earlier CUDA failure. That failure did not recur; this does not establish its root cause. The run's functional assertions passed before the separate launcher teardown issue described below. - No full RL training, checkpoint restart, or comparative throughput claim. </details> <details> <summary>Limits and follow-ups</summary> The caller coordinates complete replacement attempts. There is no partial scoring, automatic group scheduling, durable answer reuse, or change to generation/judge overlap. Legacy identities cannot isolate overlapping runs or provide completed replay. Explicit-ID replay and failure records are process-local and bounded by retention; eviction or restart is not a recovery protocol. Ordinary individual reverification is unsupported. The existing `judge_failed_only` guard bypass does not create groups or advance attempts: retained failures remain failures until the caller supplies a complete replacement with the required identity. Active-group memory limits, retention optimization, caller-minted group identities, durable outcomes, richer telemetry, and improvements to retry backoff remain follow-up work. The inherited judge-failed-only mode-guard bypass needs a separate compatibility change; documenting it does not make individual GenRM reverification supported. A Ray teardown abort was previously reproduced on both main and an earlier PR head; its native cause remains undiagnosed. In the passing job 19084624, the Gym launcher again exited with SIGABRT after the functional assertions completed. The harness stopped the remaining owned children and recorded no survivors; the separate policy server exited normally and Slurm released the allocation. This validates the GenRM failure/recovery behavior, not a clean Ray launcher shutdown. </details> ### Related PRs: what is covered and what could follow Built on NVIDIA-NeMo#2814 (Anish Mahishi), with deadline and cleanup requirements from NVIDIA-NeMo#2385 (Teodor-Dumitru Ene) and NVIDIA-NeMo#2903 (Jiacheng Xu). Contributor credit remains in commit trailers. **NVIDIA-NeMo#2385:** the timeout requirement is covered. Its partial-scoring policy is not adopted: if A/B/C arrive but D is missing, this PR fails the incomplete group. Whole-group failure is allowed by NVIDIA-NeMo#3181. Close NVIDIA-NeMo#2385 as superseded if that policy is accepted; otherwise retain/rework partial scoring as separate follow-up work with explicit missing-member and training/metric semantics. **NVIDIA-NeMo#2903:** together, NVIDIA-NeMo#2814 and this PR cover request deduplication, duplicate-disconnect safety, and failure cleanup. Three policy differences remain: - NVIDIA-NeMo#2903 permits a fresh group under the same identity after failure. Here, replacement requires a newer shared explicit attempt, or a fresh explicit ID when migrating a failed legacy group. - NVIDIA-NeMo#2903 keeps completed rewards for the server's lifetime, including legacy groups. Here, completed replay is bounded and restricted to identical responses in explicit-ID groups; legacy groups have no completed replay. - NVIDIA-NeMo#2903 starts comparing available answer pairs while other answers are still generating. Here, we preserve NVIDIA-NeMo#2814's wait-for-all schedule. Each judge call needs only two answers, so early judging remains a possible optimization; it does not require partial reward publication, and its performance benefit has not been benchmarked. After this PR merges, maintainers can close NVIDIA-NeMo#2903 as superseded if these choices are accepted, tracking desired early judging or broader replay/retry behavior separately. Alternatively, rework the existing PR against current main to focus on the remaining scope. Closing either older PR means its core problem is addressed and remaining choices are accepted or tracked, not that every proposed behavior was merged. --------- Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> Co-authored-by: Anish Mahishi <amahishi@nvidia.com> Co-authored-by: Jiacheng Xu <jiachengx@nvidia.com> Co-authored-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Glorf
added a commit
that referenced
this pull request
Sep 28, 2026
## Summary Add an opt-in `sink_window` history policy to the OSWorld Nemotron adapter: retain the earliest screenshots alongside a recent-image window, with intervening turns represented as text in chronological order. This branch includes upstream `main` at `d4c0f86c7` through a normal merge, including the NumPy metadata fix #3649, CI preflight update #3658, and sandbox lifecycle telemetry #3633. The OSWorld foundation from #2554/#3140 was consolidated and merged through #3592, so this PR no longer carries the old exact-trace/Core stack or depends on #2554. It does not require #3591. No separate issue is needed for this follow-up policy extension. ## Changes - Add configurable sink, low-water and high-water counts. Both watermarks include the sink; equal watermarks slide, while unequal watermarks provide hysteresis. - Represent non-contiguous image selections with intervals and render folded text in chronological order. - Distinguish context overflow, transport failure, truncated output, empty output and unparseable output; retain `agent_outcome_family=model_response_invalid` for compatibility. - On context overflow, reduce the recent-image window within the existing retry budget, preserve the sink/current observation, record actual shrink events, and stop when further reduction is impossible. - Preserve main's immediate model-timeout propagation. Report the terminal attempt's completion fact so an earlier malformed sample cannot hide a subsequent transport failure. The latest synchronization also annotates six deterministic SHA-256 prompt goldens that the previous secrets-detector run falsely identified as credentials. Only those literal lines are annotated; the digest values, assertions, repository scanner configuration, and baseline are unchanged. ## Design and compatibility History policy selects image/text turns; the adapter owns model protocol rendering and parsing. The runner owns horizon and runtime failure handling, the evaluator owns reward, and runtime admission owns masking. This PR does not change those ownership boundaries or add a training/evaluation switch. Fixed-window and hysteresis policy IDs and normal-path rendered prompts remain covered by deterministic golden tests, including fixed-3 and 3-10-3. Sink-window is opt-in, not a new default or a claim of improved benchmark score. Context-overflow recovery and more precise failure outcomes are intentional behavioral changes and may affect failed-run outcomes. The legacy scalar `image_window_start` accessor rejects non-contiguous plans; consumers should use `image_intervals` or per-turn decisions. Scope relative to current main remains limited to `responses_api_agents/osworld_agent/`: two implementation files, documentation, and tests. No Gym Core, dependency, deployment, sandbox-provider or model-server changes are included in the PR diff. ## Validation Re-run on the final main-synchronized tree: - **440 passed, zero skipped** across OSWorld agent, resource-server and benchmark tests; one non-failing Starlette deprecation warning. - `pre-commit run --all-files`: passed. - `uv lock --check --offline`: passed. - `git diff --check`: passed. - The same `detect-secrets==1.5.0` hook and checked-in baseline used by CI passed on all nine PR-scoped files after the inline false-positive annotations. - Existing model-timeout propagation, runner admission, prompt-parity, sink selection, overflow recovery and policy-identity regressions passed. The unchanged-overflow retry and mixed parse/transport completion bugs had been reproduced before their earlier fixes. Test invocation, using an existing compatible development environment from this checkout: ```bash export PYTHONPATH="$PWD" export NEMO_GYM_EXTRA_ROOTS="$PWD" python -m pytest \ responses_api_agents/osworld_agent/tests \ resources_servers/osworld/tests \ benchmarks/osworld/tests \ --import-mode=importlib -o addopts='' -o log_cli=false -q pre-commit run --all-files ``` The explicit source root prevents an older editable checkout from shadowing the branch under test; importlib mode avoids the shared `osworld` test-package name collision. These checks are local evidence, not a claim that the upstream CI run has completed. ## Score reproduction OSWorld 361 tasks, Nemotron-3-Nano-Omni v3 (`24e67ea0`), fixed-3 history, 100 steps, one A/B run per arm. Control = main `d4c0f86c7` + #3591 (needed for the E2B `endpoint()` on AgentEnv); treatment = control + this PR. Raw SR 44.61% → 46.62% (Δ +2.01 pp, paired 95% CI [−2.12, +6.07] pp); binary 154 → 159 of 361 (McNemar p = 0.59); 0 masked and 0 missing in both arms; identical history-policy and agent-contract IDs. No degradation. The context-overflow shrink path was not triggered in either run. --------- Signed-off-by: Jeff Peng <jepeng@nvidia.com> Co-authored-by: Michał Bień <mbien@nvidia.com>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fix the OSWorld agent's Python 3.13 dependency-resolution failure by correcting the OSWorld package metadata at its source, rather than adding another NumPy override or changing Gym's shared installer.
nv-gym.numpy>=2.1,<2.5requirements remain unchanged.The fork commit synchronizes
pyproject.toml,requirements.txt, andsetup.py: Python <3.13 retains NumPy 1.26/OpenCV 4.8; Python >=3.13 declares NumPy >=2.1,<3 with headless OpenCV >=4.10.0.84,<5. It adds 18 marker/ABI-pair regression cases. These are compatibility ranges, not experimental version locks.Why is this needed?
The scheduled server-suite failure occurs before OSWorld tests run:
#3592 introduced the agent's NumPy 2 requirement while its pinned OSWorld archive still declared
numpy<2.osworld==0.1.0is the fork package's metadata version; CI already downloads our explicit GitHub archive, not an unrelated PyPI package. The later README-only #3631 did not introduce this regression.The role's
uv.tomlNumPy override depended on explicit config loading in the still-open #3591. Current main's generated requirements install does not load that role config in this checkout layout. Fixing the fork metadata removes this particular dependency on #3591. Its other runtime/embedded-Ray changes remain separate and are not claimed fixed here.Validation
--dry-runand a Linux target added, in a fresh temporary venv and including the parent's Ray 2.56.1/OpenAI 2.44.0 requirements: 350 packages resolved.Requires-Distentries.pre-commit run --all-filesandgit diff --check: passed.The full eight-shard server suite and a fresh Linux managed-venv installation are left to CI; resolver checks are not a claim that those runs completed locally.
Compatibility and benchmark impact
The fork's
desktop_env,mm_agents, andevaluation_examplesGit tree IDs are byte-identical to the previous pinned revision. Tasks, evaluators, prompts, parser, history policy, and exact-trace behavior are unchanged. Existing Gym role runtime ranges are also unchanged; only their upstream metadata conflict is removed.Model rollout: N/A for this metadata-only fix. No new model score or training result is claimed. Other stale dependency overrides are intentionally retained; this is not a general OSWorld packaging migration.