Skip to content

docs: describe the repository layout and the nemo_gym packages - #5385

Merged
ananthsub merged 1 commit into
mainfrom
ananthsub/layout-docs
Nov 3, 2026
Merged

ananthsub merged 1 commit into
mainfrom
ananthsub/layout-docs

Conversation

@ananthsub

@ananthsub ananthsub commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

What changed and why

The package reorganization (tracking issue #5166) split nemo_gym/ into layered packages, and every PR in that stack has merged. The API reference overview still described the library as a short list of "key modules" with their class names, which no longer matches the code and goes stale whenever a feature adds a class. This PR replaces that list with a description of the repository layout and fixes other pages that pointed at code that moved.

  • API reference overview (fern/versions/latest/pages/api-reference/index.mdx): the "Key Modules" table is replaced by three sections.
    • Repository Layout says what each top-level directory holds, such as nemo_gym/, the server directories, environments/, benchmarks/ and tests/.
    • The nemo_gym Package has a layer diagram and one line per package, with the rule that a package builds only on packages below it. It also covered failure_transport, typed_exceptions and exception_inspection, which sat directly in nemo_gym/ when this PR merged; refactor(core): move the failure modules from #5072 into their packages #5536 has since moved them into core/ and utils/. It covers nemo_gym.verifiers, which sits outside the layers because it runs inside verifier sandboxes.
    • Old Import Paths says that old paths keep working with a DeprecationWarning until 0.10.0 and that scripts/migrate_imports.py rewrites them.
  • Architecture page (fern/versions/latest/pages/about/architecture.mdx): a short "Where the Code Lives" section names the server directories and links to the repository layout.
  • AGENTS.md: the architecture section names the modules that define BaseServer, SimpleServer, SimpleResourcesServer, SimpleResponsesAPIModel and SimpleResponsesAPIAgent, lists the package order, and points to the import-linter contract in pyproject.toml that enforces it.
  • Ray FAQ (fern/versions/latest/pages/reference/faq.mdx): the page named nemo_gym/cli/env.py and server_utils.py. A run starts or joins Ray from RunHelper.start() in nemo_gym/orchestration/server_supervisor.py, and each server process connects from SimpleServer.run_webserver() in nemo_gym/server_runtime/base_server.py, which calls the helpers in nemo_gym/server_runtime/ray_support.py.
  • Hugging Face download page (fern/versions/latest/pages/data/download-huggingface.mdx): the source table gave line ranges in nemo_gym/config/config_types.py that now land on the upload config. The table names DownloadJsonlDatasetHuggingFaceConfig, its check_output_path() validator, and download_hf_dataset_as_jsonl() instead, so it stays correct as the files change.
  • Code samples: a sample is code outside nemo_gym/, so it imports each name from its package rather than from the module that defines it. The resources server tutorials, aggregate metrics, model call capture, sandbox provider and telemetry extension samples now import from nemo_gym.server_runtime, nemo_gym.rollouts, nemo_gym.sandbox and nemo_gym.telemetry, and the prose next to the sandbox and telemetry samples names the package too. nemo_gym.verifiers.files in the Harbor tasks page stays, because the verifier helpers are a standard-library-only package with no public API of their own.

Only the bleeding-edge docs under fern/versions/latest/ change, and the frozen fern/versions/v*/ snapshots stay as they are.

Validation

  • cd fern && npm run check (fern check) reports no errors. Its one warning is the missing-redirects check, which Fern skips without authentication.
  • The layer diagram renders with mermaid-cli.
  • Every name that a Python code sample under fern/versions/latest/pages/ imports from nemo_gym imports on this branch with deprecation warnings raised as errors.
  • Every nemo_gym file path, module path and GitHub source link that those pages cite exists on this branch, and each class or function named next to a module is defined in that module or its package.
  • pre-commit run --files on the changed files passes.

Rollout evidence: N/A, docs only.

@ananthsub ananthsub added documentation Improvements to documentation area:core Shared APIs, servers, telemetry, health, and registries labels Oct 30, 2026
@nemo-automation-bot
nemo-automation-bot Bot requested review from a team October 30, 2026 20:15
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

@ananthsub
ananthsub force-pushed the ananthsub/layout-cli branch from 856de32 to e918648 Compare October 30, 2026 21:03
@ananthsub
ananthsub requested review from a team as code owners October 30, 2026 21:03
@ananthsub
ananthsub force-pushed the ananthsub/layout-docs branch from c5ea278 to 58c586c Compare October 30, 2026 21:04
@svcnvidia-nemo-ci svcnvidia-nemo-ci added the needs-review PR is ready for code review and waiting on a reviewer label Oct 30, 2026
@ananthsub
ananthsub force-pushed the ananthsub/layout-cli branch from e918648 to f209da8 Compare October 30, 2026 23:19
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Nov 2, 2026
@ananthsub
ananthsub force-pushed the ananthsub/layout-cli branch 3 times, most recently from 42b8ce6 to 32cfec4 Compare November 3, 2026 17:01
@ananthsub
ananthsub force-pushed the ananthsub/layout-docs branch from 58c586c to b9dc200 Compare November 3, 2026 18:02
@ananthsub
ananthsub force-pushed the ananthsub/layout-cli branch from 6eba755 to 96de085 Compare November 3, 2026 18:08
@ananthsub
ananthsub force-pushed the ananthsub/layout-docs branch from b9dc200 to 7a39901 Compare November 3, 2026 18:17
@ananthsub
ananthsub force-pushed the ananthsub/layout-cli branch 2 times, most recently from 3bcfad5 to 9616fd2 Compare November 3, 2026 18:23
@ananthsub
ananthsub force-pushed the ananthsub/layout-docs branch from 7a39901 to 45f16b3 Compare November 3, 2026 18:23
@ananthsub
ananthsub force-pushed the ananthsub/layout-cli branch from 9616fd2 to a744fb6 Compare November 3, 2026 18:27
@ananthsub
ananthsub force-pushed the ananthsub/layout-docs branch from 45f16b3 to 56ed5ee Compare November 3, 2026 18:28
@ananthsub
ananthsub force-pushed the ananthsub/layout-cli branch from a744fb6 to d5ee75a Compare November 3, 2026 18:28
Base automatically changed from ananthsub/layout-cli to main November 3, 2026 18:33
@ananthsub
ananthsub force-pushed the ananthsub/layout-docs branch from 56ed5ee to de96e27 Compare November 3, 2026 18:33
The API reference overview listed a few "key modules" by their class names.
That list no longer matches the code after `nemo_gym/` was split into layered packages, and class
lists go stale as features are added.

The overview now has two sections:
- Repository Layout says what each top-level directory holds.
- The nemo_gym Package has a layer diagram and one line per package. It also covers the three
  modules that sit directly in `nemo_gym/` and the `verifiers` helpers that sit outside the layers.
  It ends with the import rule and how long old import paths keep working.

The architecture page links to the repository layout.
The architecture section of `AGENTS.md` names the modules that define the server base classes and
lists the package order.

Other pages pointed at code that moved or that never held what they described:
- The Ray FAQ named `nemo_gym/cli/env.py` and `server_utils.py`. A run starts Ray from
  `RunHelper.start()` in `nemo_gym/orchestration/server_supervisor.py`. Each server process
  connects from `SimpleServer.run_webserver()` in `nemo_gym/server_runtime/base_server.py`.
- The Hugging Face download page gave line ranges that now land on the upload config. It names
  the download config, its validator, and the download function instead.
- Code samples imported names from the module that defines them. Samples are code outside
  `nemo_gym/`, so they now import from each package, such as `nemo_gym.server_runtime`,
  `nemo_gym.rollouts`, `nemo_gym.sandbox` and `nemo_gym.telemetry`.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
@ananthsub
ananthsub force-pushed the ananthsub/layout-docs branch from 4c161ab to 8e7a211 Compare November 3, 2026 20:10
@ananthsub ananthsub changed the title docs: describe the reorganized nemo_gym package layout docs: describe the repository layout and the nemo_gym packages Nov 3, 2026
@ananthsub
ananthsub merged commit 6ff9fc6 into main Nov 3, 2026
13 checks passed
@ananthsub
ananthsub deleted the ananthsub/layout-docs branch November 3, 2026 20:16

This branch was previously deployed

1 inactive deployment
public — 8e7a211b Deployed Nov 4, 2026 by copy-pr-bot[bot] via release / finalize / notify #9750
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core Shared APIs, servers, telemetry, health, and registries documentation Improvements to documentation needs-review PR is ready for code review and waiting on a reviewer sla:triage-overdue Review assignment is over the one-business-day SLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants