Repository navigation
feat: add standalone Gym Docker container - #2161
Merged
Merged
Conversation
- Add docker/Dockerfile: CUDA base (cuda-dl-base 26.03), Python 3.12 via uv, Gym core deps via locked uv sync, git pre-installed. Supports optional VLLM_VERSION build arg and placeholder NEMO_GYM_PREFETCH_CONFIGS arg for future resource-server prefetching. - Add .github/workflows/build-container.yml: triggers container build via the existing _build_container.yml reusable workflow on push/PR to main when Dockerfile, pyproject.toml, or uv.lock change. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
NVIDIA self-hosted runners (linux-amd64-cpu16) block workflows triggered by pull_request events per org security policy. Container builds only run on push to main and workflow_dispatch. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
Contributor
Author
|
/claude review |
- Add ENV VIRTUAL_ENV=/opt/nemo_gym_venv in hermetic stage so uv pip install (used for optional vLLM) correctly targets the project venv. uv pip ignores UV_PROJECT_ENVIRONMENT; it needs VIRTUAL_ENV or --python. - Remove NEMO_GYM_PREFETCH_CONFIGS arg and dead gym env prefetch block. The prefetch CLI subcommand doesn't exist yet; passing the arg would fail the build. Will re-add once the command is implemented in Gym. - Add --link-mode symlink to hermetic uv sync, matching RL's pattern. - Remove duplicate WORKDIR in release stage. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
vLLM version should be locked via pyproject.toml + uv.lock and installed with uv sync --extra vllm --locked, not pinned by the caller at build time. Remove the VLLM_VERSION ARG and uv pip install block entirely. Will re-add as a proper uv extra once the vllm optional dependency is wired in pyproject.toml. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
- pyproject.toml: add [vllm] optional-dependencies extra with vllm as dep - uv.lock: resolve vllm==0.26.0 and its transitive deps (293 packages total) - docker/Dockerfile: install --extra vllm in both the hermetic dep-sync layer and the release project-install layer so vLLM is baked in and version-locked via the lockfile (not caller-controlled) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
Contributor
Author
|
/claude review |
Gym uses static versioning (attr = "nemo_gym.__version__"), not setuptools-scm, so git tag history is not needed at build time. The unshallow step and its comment were copied verbatim from RL where setuptools-scm is used. Remove both. Also remove the redundant WORKDIR /opt/nemo-gym at the end of the release stage — it is inherited from hermetic and already set before the uv sync runs. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
Without --build-context nemo-gym=<path>, the hermetic stage fetches pyproject.toml and uv.lock from GitHub main (via ADD --keep-git-dir), missing changes on the PR branch (e.g. the vllm optional extra). FW CI templates check out the repo at path: github.run_id, so the correct build context is nemo-gym=<run_id>/ relative to the workspace. Add build-contexts input to _build_container.yml and forward it through. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
ananthsub
reviewed
Jul 28, 2026
vLLM version sync: - pyproject.toml [vllm] extra: vllm==0.26.0 + flashinfer-python==0.6.14 (exact version vllm 0.26.0 specifies; ensures pre-compiled kernels are used to avoid JIT compilation on first generation / slow step time) - responses_api_models/local_vllm_model/setup.py: 0.20.0 → 0.26.0 - responses_api_models/genrm_model/setup.py: 0.20.0 → 0.26.0 (must stay in sync per the comment in genrm setup.py) - uv.lock: re-locked, 293 packages (flashinfer-python 0.6.14 was already a transitive dep of vllm 0.26.0, no net change) Prefetch: - docker/Dockerfile: add NEMO_GYM_PREFETCH_SERVERS build arg Pass a space-separated list of server paths relative to the repo root. For each server, creates its isolated .venv via uv sync at build time, so the venv is ready at runtime without any download overhead. Example: --build-arg NEMO_GYM_PREFETCH_SERVERS="responses_api_models/local_vllm_model" Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
uv sync reads [project.dependencies] from pyproject.toml but server deps
(vllm, uvicorn, hf_transfer, etc.) are declared in setup.py install_requires
under dynamic = ["dependencies"] — uv sync never sees them and creates an
empty venv.
Fix mirrors gym env start (setup_command.py:141):
uv pip install -e . runs setup.py and picks up install_requires correctly
--python .venv/bin/python targets the server .venv, not UV_PROJECT_ENVIRONMENT
which is globally set to /opt/nemo_gym_venv
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Kajal Jain <kajalj@nvidia.com>
After a successful build, the 'test' job pulls the container from ACR and runs two checks: - gym --help: verifies the CLI entrypoint works - import check: asserts vllm==0.26.0 and flashinfer-python==0.6.14 are installed at the expected versions (uses importlib.metadata to avoid triggering CUDA initialisation on a CPU runner) Also expose container-uri output from _build_container.yml so build-container.yml can reference the built image URI. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
ENTRYPOINT is ["gym"] so 'docker run <img> gym --help' invokes 'gym gym --help' which fails. Correct form is 'docker run <img> --help'. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
Adds flashinfer-python at the exact version vllm 0.26.0 requires, matching the pyproject.toml [vllm] extra. Ensures per-server venvs use pre-compiled CUDA kernels and avoid JIT compilation on first generation. Mac path unchanged (vllm==0.11.0, no flashinfer). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
Switch to ubuntu-latest (GitHub-hosted) so the build runs during PRs as a required check. Self-hosted runners block pull_request events, so ubuntu-latest is needed for PR validation. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
Signed-off-by: Kajal Jain <kajalj@nvidia.com>
chtruong814
previously approved these changes
Jul 30, 2026
ananthsub
previously approved these changes
Jul 30, 2026
Per review feedback (Ananth), prefetch should work like RL's prefetch_venvs.py — accepting Gym configs and running a dry install of deps. Removing the current incomplete implementation to unblock shipping the container; proper prefetch via 'gym env prefetch' will land in a follow-up PR. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
…/Gym into add-docker-container
Restores the path-based prefetch build arg removed in the previous commit. A follow-up PR (branching from this one) will replace this with a config-based 'gym env prefetch' command similar to RL's prefetch_venvs.py. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
kajalj22
added a commit
that referenced
this pull request
Aug 3, 2026
Placeholder for 'gym env prefetch' implementation. Addresses review feedback from #2161. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>
…#2255) ## Summary Follow-up to #2161 (standalone Gym container). Implements `gym env prefetch` — a proper config-driven CLI command for pre-warming server venvs at container build time, similar to how RL's `prefetch_venvs.py` works. ## What this replaces PR #2161 has `NEMO_GYM_PREFETCH_SERVERS` which takes space-separated directory paths. This PR replaces it with `NEMO_GYM_PREFETCH_CONFIGS` which takes Gym config files — the same format as `gym env start`. This is the approach Ananth described in the review. ## How it works ``` gym env prefetch "+config_paths=[config1.yaml,config2.yaml]" ``` 1. Parses configs the same way `gym env start` does 2. For each server found in the config, calls `setup_env_command()` — creates the venv and installs deps serially 3. Exits — **no Ray initialised, no server processes started** Serial installation reduces uv cache size (same reasoning as `dry_run` mode in `RunHelper.start`). ## Dockerfile usage ```dockerfile # Pre-warm venvs for your server configs at build time: docker buildx build \ --build-arg NEMO_GYM_PREFETCH_CONFIGS="responses_api_models/local_vllm_model/configs/local_vllm_model.yaml,resources_servers/math/configs/math.yaml" \ -f docker/Dockerfile . ``` ## Changes - `nemo_gym/cli/env.py` — `gym env prefetch` command - `docker/Dockerfile` — `NEMO_GYM_PREFETCH_CONFIGS` replaces `NEMO_GYM_PREFETCH_SERVERS` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Kajal Jain <kajalj@nvidia.com> Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
ananthsub
approved these changes
Aug 4, 2026
kajalj22
enabled auto-merge (squash)
August 4, 2026 19:08
kajalj22
disabled auto-merge
August 4, 2026 22:33
kajalj22
added a commit
that referenced
this pull request
Aug 5, 2026
…2334) beep boop [🤖]: Hi @kajalj22 👋, we've cherry picked #2161 into for you! 🚀 Please review and approve this cherry pick by your convenience! Signed-off-by: Kajal Jain <kajalj@nvidia.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: kajalj22 <kajalj@nvidia.com> Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
OlegSudakov
pushed a commit
to OlegSudakov/Gym
that referenced
this pull request
Aug 7, 2026
## Standalone NeMo-Gym container
```
FROM cuda-dl-base:26.03 (ubuntu + CUDA, no NGC pytorch)
│
▼
[base stage]
Python 3.12, uv, git
│
▼
[hermetic stage]
uv sync --extra vllm --locked
├── nemo-gym core (ray, fastapi, aiohttp, mlflow …)
├── vllm == 0.26.0
└── flashinfer-python == 0.6.14 (pre-compiled kernels)
│
▼
[release stage]
install nemo-gym project
optional: NEMO_GYM_PREFETCH_SERVERS → per-server .venvs
entrypoint: gym
```
**Per-server venvs** (e.g. `local_vllm_model`, `genrm_model`) are
separate isolated environments created by `gym env start` at runtime via
`uv pip install -e .`. Pass `NEMO_GYM_PREFETCH_SERVERS` to bake them in
at build time instead.
## CI
`build-container.yml` runs on `pull_request` and `push` to main (when
`Dockerfile`/`pyproject.toml`/`uv.lock` change) using `ubuntu-latest`.
Validates the Docker build only — no push, no registry.
## Changes
- `docker/Dockerfile` — 4-stage build, vllm + flashinfer locked
- `pyproject.toml` — `[vllm]` optional extra (`vllm==0.26.0`,
`flashinfer-python==0.6.14`)
- `uv.lock` — locked packages
- `local_vllm_model/setup.py`, `genrm_model/setup.py` — flashinfer
pinned alongside vllm
- `.github/workflows/build-container.yml` — build-only CI, triggers on
PR and push to main
## Dispatch
https://github.com/NVIDIA-NeMo/Gym/actions/workflows/build-container.yml
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Kajal Jain <kajalj@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
linj-glitch
pushed a commit
to linj-glitch/Gym
that referenced
this pull request
Oct 15, 2026
…DIA-NeMo#4639) ## What does this PR do? Three pages under `fern/versions/latest/pages` cite `resources_servers/*/configs/*.yaml` files that do not exist on main. This PR points each one at the file that replaced it. It also adds a docs test so the next rename that leaves a citation behind fails CI. No issue or NVBug: found by scanning the latest pages for cited config paths and checking each one exists: ```bash grep -rhoE 'resources_servers/[a-z0-9_]+/configs/[A-Za-z0-9_.-]+\.yaml' fern/versions/latest/pages \ | sort -u | while read f; do [ -f "$f" ] || echo MISSING $f; done ``` | Page | Cited path | Now | Why | | --- | --- | --- | --- | | `reference/cli-commands.mdx` (`gym env prefetch` example) | `resources_servers/math/configs/math.yaml` | `resources_servers/math_with_judge/configs/math_with_judge.yaml` | `resources_servers/math/` never existed. The example came from the `prefetch` docstring added in NVIDIA-NeMo#2161 and was copied into the page by NVIDIA-NeMo#4093. `math_with_judge.yaml` is the math config the same page already uses for `ng_e2e_collect_rollouts`. The docstring in `nemo_gym/cli/env.py` gets the same fix. | | `contribute/rl-framework-integration/gym-rl-framework-integration-success-criteria.mdx` | `math_with_judge/configs/bytedtsinghua_dapo17k.yaml` (pinned to `299e8c04`) | `math_with_judge/configs/dapo17k.yaml` on `main` | NVIDIA-NeMo#859 deleted `bytedtsinghua_dapo17k.yaml` and moved its DAPO17k train and AIME24 validation sets into `dapo17k.yaml`. The pinned permalink still resolves, but it points at a config that main no longer ships. The other links in this page's tables already use `main`. | | `reference/faq.mdx` (4 dataset upload/migrate examples) | `resources_servers/multineedle/configs/multineedle.yaml` | `resources_servers/example_multi_step/configs/example_multi_step.yaml` | NVIDIA-NeMo#291 renamed `multineedle` to `example_multi_step`. The upload command reads `domain` from this config, and `example_multi_step.yaml` sets it. | Versioned snapshots under `fern/versions/v*/` are frozen and left as they shipped. ### Test `tests/unit_tests/test_fern_docs_links.py::test_cited_resources_server_configs_exist` scans every latest page for `resources_servers/<name>/configs/<file>.yaml` and asserts that each file exists. It skips `my_weather_tool`, which `environment-tutorials/single-step-environment.mdx` has the reader create. It also asserts that it found at least one citation, so a broken regex can't make it pass vacuously. ## Before the fix Gym `974d1a921` (main). The new test fails (`1 failed`) and reports these six citations: ``` fern/versions/latest/pages/contribute/rl-framework-integration/gym-rl-framework-integration-success-criteria.mdx:38: resources_servers/math_with_judge/configs/bytedtsinghua_dapo17k.yaml fern/versions/latest/pages/reference/cli-commands.mdx:695: resources_servers/math/configs/math.yaml fern/versions/latest/pages/reference/faq.mdx:41: resources_servers/multineedle/configs/multineedle.yaml fern/versions/latest/pages/reference/faq.mdx:53: resources_servers/multineedle/configs/multineedle.yaml fern/versions/latest/pages/reference/faq.mdx:91: resources_servers/multineedle/configs/multineedle.yaml fern/versions/latest/pages/reference/faq.mdx:108: resources_servers/multineedle/configs/multineedle.yaml ``` The documented `gym env prefetch` example, run with venv installs stubbed out so that only config resolution and server discovery run (`policy_model_name` is normally supplied by `env.yaml`): ``` $ gym env prefetch "+config_paths=[responses_api_models/local_vllm_model/configs/local_vllm_model.yaml,resources_servers/math/configs/math.yaml]" ++policy_model_name=Qwen/Qwen3-4B-Instruct-2507 Error: config_paths entry 'resources_servers/math/configs/math.yaml' was not found. Looked in: - <Gym>/resources_servers/math/configs/math.yaml Check the path is spelled correctly and is relative to your working directory, an extra root (NEMO_GYM_EXTRA_ROOTS / --search-dir), or the Gym install root. exit=1 ``` ## After the fix ``` $ gym env prefetch "+config_paths=[responses_api_models/local_vllm_model/configs/local_vllm_model.yaml,resources_servers/math_with_judge/configs/math_with_judge.yaml]" ++policy_model_name=Qwen/Qwen3-4B-Instruct-2507 Pre-warming venv for: responses_api_models/local_vllm_model Pre-warming venv for: resources_servers/math_with_judge Pre-warming venv for: responses_api_agents/simple_agent Pre-warming venv for: environment_servers/legacy_agent Prefetch complete. exit=0 ``` `gym env resolve --config` now finds `dapo17k.yaml` and `example_multi_step.yaml`. It then stops at the usual unset-required-values check (`policy_model` / judge name), the same as any config resolved without an `env.yaml`. ## Validation - `pytest tests/unit_tests/test_fern_docs_links.py tests/unit_tests/test_opensandbox_documented_config.py`: 25 passed, 21 subtests passed. - `pytest tests/unit_tests/test_cli_main.py`: passed (the `env.py` change only touches a docstring). - `pre-commit run --all-files`: all hooks pass. - Rollouts: N/A. The only change outside docs and tests is the example text in one docstring. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: qiaochuz <qiaochuz@nvidia.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Standalone NeMo-Gym container
Per-server venvs (e.g.
local_vllm_model,genrm_model) are separate isolated environments created bygym env startat runtime viauv pip install -e .. PassNEMO_GYM_PREFETCH_SERVERSto bake them in at build time instead.CI
build-container.ymlruns onpull_requestandpushto main (whenDockerfile/pyproject.toml/uv.lockchange) usingubuntu-latest. Validates the Docker build only — no push, no registry.Changes
docker/Dockerfile— 4-stage build, vllm + flashinfer lockedpyproject.toml—[vllm]optional extra (vllm==0.26.0,flashinfer-python==0.6.14)uv.lock— locked packageslocal_vllm_model/setup.py,genrm_model/setup.py— flashinfer pinned alongside vllm.github/workflows/build-container.yml— build-only CI, triggers on PR and push to mainDispatch
https://github.com/NVIDIA-NeMo/Gym/actions/workflows/build-container.yml
🤖 Generated with Claude Code