feat(cli): make built-in assets work from a wheel install / external cwd by wprazuch · Pull Request #1805 · NVIDIA-NeMo/Gym

wprazuch · 2026-06-29T09:04:13Z

Problem

A wheel/PyPI install of NeMo Gym shipped almost no built-in assets and resolved component names only against the current working directory. So from any directory other than a repo checkout:

gym list environments returned nothing (the environments/ tree wasn't packaged at all).
gym env start --resources-server <name> / --environment <name> / --benchmark <name> failed (the install root was never searched).

This is the packaging / install-root slice of epic #1205 (criteria C2 run-by-name/no-internal-paths, C5 external-dependency/path-safety).

Root cause

pyproject.toml: packages.find omitted environments, and there was no package-data/MANIFEST, so the 388 config.yaml + 254 data/*.jsonl + READMEs + prepare.py never shipped in the wheel (only nemo_gym/resources/*.py did).
nemo_gym/cli/main.py: _asset_config_path searched [WORKING_DIR, *--search-dir], and WORKING_DIR == cwd for wheel installs — so the install root (PARENT_DIR, i.e. site-packages) was never searched.

Changes

Packaging (pyproject.toml)

Add environments / environments.* to packages.find.
Ship built-in assets via package-data globs: every *.yaml, README.md, prepare.py, and small data/example*.jsonl across resources_servers, responses_api_agents, responses_api_models, environments, benchmarks. Bulk train/validation JSONL is intentionally excluded — those are fetched on demand via a dataset source: / --download, keeping the wheel lean.
Require setuptools>=64 for reliable recursive ** package-data globs.

Resolution (nemo_gym/cli/main.py)

_asset_config_path now searches the install root (PARENT_DIR) in addition to the cwd / --search-dir roots, deduped by resolved path. PARENT_DIR is the install root in both editable (repo) and wheel (site-packages) layouts, so built-ins resolve by name regardless of cwd. Editable-from-repo still dedupes to a single root (no behavior change). A user asset that shadows a built-in of the same name remains an explicit "ambiguous" error (use --config).

Tests (tests/unit_tests/test_cli_main.py)

New TestInstallRootResolution: built-in resolves from the install root when cwd differs; user-cwd asset resolves; same-name shadow is ambiguous; editable single-root isn't self-ambiguous.

Verification

Wheel inspection: ships 12/12 environments, 175 resources-server + 34 model + 31 agent configs, 212 example JSONL, 0 train/validation JSONL.
End-to-end (the repro): in a clean venv with only the wheel installed, run from /tmp → discover_environments() returns 12 (was 0) and --environment blackjack resolves to site-packages/environments/blackjack/config.yaml.
ruff check/format clean; test_cli_main.py / test_registry.py / test_cli.py / test_cli_agents.py pass.

Follow-ups (out of scope here)

Other cwd-anchored sites from the audit remain for a follow-up PR: train_data_utils (jsonl_fpath), prompt.py, cli/env.py test globs, cli/setup_command.py PYTHONPATH.

…rk from any cwd A wheel install shipped almost no built-in assets and resolved component names only against the cwd, so `gym list environments` was empty and `gym env start --resources-server <name>` failed from any external directory (epic #1205, criteria C2/C5; nvbug 6383277). Packaging (pyproject.toml): - add `environments` to packages.find - ship every config.yaml, README, prepare.py, and small data/example*.jsonl across the asset trees via package-data globs; bulk train/validation JSONL is still fetched on demand via a dataset `source:` / `--download` - require setuptools>=64 for recursive `**` package-data globs Resolution (nemo_gym/cli/main.py): - `_asset_config_path` now searches the install root (PARENT_DIR) in addition to the cwd / --search-dir roots, deduped by resolved path. PARENT_DIR is the install root in both editable and wheel layouts, so built-ins resolve by name regardless of cwd. Editable-from-repo dedupes to a single root (no change). Verified: wheel ships 12/12 environments + configs + example data (0 bulk JSONL); from an external cwd a wheel install discovers all 12 environments and resolves `--environment blackjack` to the install-root config. Signed-off-by: Wojciech Prazuch <wprazuch@nvidia.com>

copy-pr-bot · 2026-06-29T10:10:34Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

…ackage-data globs - explain why the floor is bumped: recursive `**` package-data globs need setuptools>=62.3 (pinned 64 for margin); build-time only, not a runtime constraint - collapse the per-tree package-data entries (which repeated the same `**/*.yaml`/`**/*.yml`/`**/README.md` globs) into a single "*" all-packages key Wheel contents verified unchanged: 12 environments, 175 resources-server configs, 254 example JSONL, 0 train/validation JSONL. Signed-off-by: Wojciech Prazuch <wprazuch@nvidia.com>

…le-finder The package-data globs added earlier were a no-op: built-in assets ship into the wheel via the setuptools-scm VCS file-finder (git-tracked files), not package-data. Verified by building with the globs removed — the wheel still ships all 12 environment configs, 254 example JSONL, and 175 resources-server configs, and a clean-venv wheel install runs `gym list environments` (12) and `gym env validate` from an unrelated cwd. - remove the package-data globs (incl. the "*" key, which setuptools does not honor as an all-packages key anyway — that is the empty-string key) and the `setuptools>=64` bump they needed for recursive `**`; revert floor to >=61 - the real fix remains adding `environments` to packages.find - document that assets ship via packages.find + the VCS file-finder, and that bulk train/validation JSONL is excluded because it is not committed Signed-off-by: Wojciech Prazuch <wprazuch@nvidia.com>

Follow-up to the packaging/install-root work (#1805). Relative dataset and prompt-config paths now resolve against the cwd first, then the Gym install root, so built-in files referenced by their repo-relative path (e.g. `resources_servers/<env>/data/example.jsonl`) work from any cwd, including a wheel install (epic #1205, criterion C5; nvbug 6383277). - add `nemo_gym.resolve_under_cwd_or_install(path)`: a shared read-path resolver (absolute as-is; else cwd, then install root), mirroring config_paths resolution - prompt.py: `--prompt-config` now resolves cwd-first then install root (was install-root only, so a user's own relative prompt config was never found) - train_data_utils: dataset existence-check and collation reads use the resolver so a bundled example dataset resolves from an external cwd; metrics-write paths intentionally stay cwd-relative (never write into the install root) Tests: TestResolveUnderCwdOrInstall (absolute / cwd-preferred / install-fallback / missing) plus a cwd-first prompt-config test. Deferred to a follow-up (gym env test from an external cwd: test_all globs + setup_command PYTHONPATH) — that path is # pragma: no cover and needs reworking the test-execution resolution, so it rides with the gym env test/validate work. Signed-off-by: Wojciech Prazuch <wprazuch@nvidia.com>

## Problem Follow-up to #1805. Relative dataset and prompt-config paths were resolved only against the current working directory (dataset reads) or only against the install root (prompt configs). So built-in files referenced by their repo-relative path didn't resolve from an arbitrary cwd, and a user's own relative `--prompt-config` was never found. Part of epic #1205 (criterion **C5**, external-dependency/path-safety). ## Changes - **`nemo_gym/__init__.py`** — add `resolve_under_cwd_or_install(path)`: a shared **read-path** resolver. Absolute → unchanged; relative → tried under the cwd (the user's project) first, then the Gym install root (`PARENT_DIR`, where built-ins live in both editable and wheel installs). Mirrors `config_paths` resolution. Documented as read-only — never for write targets. - **`nemo_gym/prompt.py`** — `_resolve_path` now resolves **cwd-first then install root** (was install-root only). A user can pass a relative `--prompt-config` from their own project, and built-in prompt YAMLs still resolve by repo-relative path from any cwd. - **`nemo_gym/train_data_utils.py`** — the dataset **existence-check** and **collation read** use the resolver, so a bundled `data/example.jsonl` resolves from an external cwd. The **metrics-write** paths intentionally stay cwd-relative — writing `_metrics.json` into the install root (site-packages) would be wrong. ## Tests - `TestResolveUnderCwdOrInstall`: absolute returned unchanged; cwd preferred over install root; falls back to install root; missing returns the cwd candidate. - `test_relative_path_resolves_to_cwd_first` for prompt configs. - Updated one `train_data_utils` mock that matched the old relative-string `open()` argument. ruff clean; prompt + train_data_utils + cli_main + registry suites pass (only the 3 pre-existing `TestDidYouMean` argparse failures remain, unrelated). ## Scope / follow-up Complements #1805 (which packages built-in assets incl. example data into the wheel). The `gym env test`-from-an-external-cwd plumbing (`test_all` globs + `setup_command` PYTHONPATH) is **intentionally deferred** — that path is `# pragma: no cover` and requires reworking the test-execution resolution (`TestConfig` asserts a relative entrypoint; `_test_single` does `cd {dir_path}` relative to cwd), so it rides with the `gym env test`/validate hardening work. --------- Signed-off-by: Wojciech Prazuch <wprazuch@nvidia.com>

Resolve PR #1673 conflicts with main (now includes #1804/#1805/#1806). The gym env compose work coexists with #1805's install-root _asset_config_path and the agent asset selector in nemo_gym/cli/main.py; both `env compose` and `env validate` remain registered. Auto-merge verified: no conflict markers, 167 unit tests pass (only the pre-existing TestDidYouMean argparse failures), and `gym env compose --benchmark gsm8k --agent simple_agent` + `gym env validate` both work. Signed-off-by: Wojciech Prazuch <wprazuch@nvidia.com>

…ition (RFC M3) Compose a benchmark with a swapped agent harness and/or edited benchmark-dataset params, without writing a bespoke per-combination config (epic #1205, criterion C3): - nemo_gym/config_composer.py: compose(), substitute_agent() (wires the env's resources_server/model_server/datasets into the swapped agent, filling a generic agent's mandatory resources_server.name), find_agent_block_key, substitute_dataset_params, _validate_no_mandatory_placeholders (scans for remaining ??? WITHOUT resolving interpolations), and a composability guard (self-contained Pattern B agents rejected). - gym env compose command + an agent asset selector in the unified CLI. - docs: document gym env compose in the CLI reference. Rebased onto current main (composes with #1805/#1806/#1807): #1807's env.py install-root resolution + validate are preserved; gym env compose and gym env validate coexist as separate commands. Verified: compose --benchmark gsm8k --agent simple_agent wires resources_server; validate works; env-test resolution intact. Signed-off-by: Wojciech Prazuch <wprazuch@nvidia.com>

…ition (RFC M3) Compose a benchmark with a swapped agent harness and/or edited benchmark-dataset params, without writing a bespoke per-combination config (epic #1205, criterion C3): - nemo_gym/config_composer.py: compose(), substitute_agent() (wires the env's resources_server/model_server/datasets into the swapped agent, filling a generic agent's mandatory resources_server.name), find_agent_block_key, substitute_dataset_params, _validate_no_mandatory_placeholders (scans for remaining ??? WITHOUT resolving interpolations), and a composability guard (Pattern B rejected). - gym env compose command + an agent asset selector in the unified CLI. - docs: document gym env compose in the CLI reference. Rebased onto current main (composes with #1805/#1806/#1807): #1807's env.py install-root resolution + validate preserved; gym env compose and gym env validate coexist. Verified: compose --benchmark gsm8k --agent simple_agent wires resources_server; validate works; env-test resolution intact. Signed-off-by: Wojciech Prazuch <wprazuch@nvidia.com>

wprazuch requested a review from a team as a code owner June 29, 2026 09:04

copy-pr-bot Bot temporarily deployed to public June 29, 2026 09:04 Inactive

copy-pr-bot Bot temporarily deployed to public June 29, 2026 09:06 Inactive

This was referenced Jun 29, 2026

feat: resolve built-in data & prompt files from the install root #1806

Merged

feat: clean config errors + run/test built-in servers from any cwd #1807

Merged

docs: use the unified gym CLI and name/flavor selectors across docs #1808

Closed

wprazuch force-pushed the wprazuch/gym-install-packaging branch from 9b9dba8 to 29c8452 Compare June 29, 2026 10:10

wprazuch added 2 commits June 29, 2026 12:25

marta-sd approved these changes Jun 29, 2026

View reviewed changes

ko3n1g approved these changes Jun 29, 2026

View reviewed changes

wprazuch merged commit ca19825 into main Jun 29, 2026
16 checks passed

wprazuch deleted the wprazuch/gym-install-packaging branch June 29, 2026 11:30

wprazuch mentioned this pull request Jun 29, 2026

fix: gym dataset collate creates the artifact dir when missing (run-from-wheel) #1811

Merged

Navigation Menu

Repository navigation

feat(cli): make built-in assets work from a wheel install / external cwd - #1805

feat(cli): make built-in assets work from a wheel install / external cwd#1805
wprazuch merged 3 commits into
mainfrom
wprazuch/gym-install-packaging

wprazuch commented Jun 29, 2026 •

edited

Loading

Uh oh!

copy-pr-bot Bot commented Jun 29, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

wprazuch commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Root cause

Changes

Verification

Follow-ups (out of scope here)

Uh oh!

copy-pr-bot Bot commented Jun 29, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wprazuch commented Jun 29, 2026 •

edited

Loading