Skip to content

refactor(agents): move the agent server modules into nemo_gym/agents - #5228

Merged
ananthsub merged 1 commit into
mainfrom
ananthsub/layout-agents
Nov 2, 2026
Merged

ananthsub merged 1 commit into
mainfrom
ananthsub/layout-agents

Conversation

@ananthsub

@ananthsub ananthsub commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

What and why

Moves what agent servers share into a new nemo_gym/agents/ package: the agent server base, agent outcomes and activations, sandbox sessions and the tools agents call in sandboxes, and skills.

Where this PR sits

The stack lands bottom-up; each PR's base is the branch of the PR before it, so each diff shows only that PR's change.

flowchart LR
  compat["#5167<br/>Old import paths<br/>keep working"]
  linter["#5168<br/>import-linter<br/>layer contract"]
  prep["#5169<br/>Remove path and<br/>name couplings"]
  config["#5189<br/>Move config/"]
  utils["#5195<br/>Move utils/"]
  core["#5203<br/>Move core/"]
  client["#5206<br/>Move client/,<br/>start telemetry/"]
  trajectory["#5209<br/>Move trajectory/"]
  verification["#5210<br/>Move verification/"]
  metrics["#5211<br/>Move metrics/"]
  artifacts["#5212<br/>Move artifacts/"]
  srmods["#5214<br/>Move server_runtime/<br/>modules"]
  su["#5215<br/>Split server_utils"]
  model["#5224<br/>Move models/<br/>modules"]
  modelbase["#5225<br/>Split model<br/>call capture"]
  agents["#5228<br/>Move agents/"]
  data["#5229<br/>Move data/"]
  harbor["#5230<br/>Split the<br/>Harbor task CLI"]
  rollouts["#5231<br/>Move rollouts/"]
  rsplit["#5232<br/>Split the<br/>rollout collector"]
  testing["#5233<br/>Dissolve testing/"]
  orch["#5234<br/>Move orchestration/<br/>modules"]
  sup["#5235<br/>Split the<br/>server supervisor"]
  eval["#5244<br/>Move eval/"]
  env["#5298<br/>Move environment/<br/>registries"]
  cli["#5300<br/>Retire the<br/>cli_setup_command shim"]
  docs["#5385<br/>Describe the<br/>package layout"]
  owners["next<br/>CODEOWNERS:<br/>per-package groups"]
  remove["0.10.0<br/>Remove the<br/>old paths"]
  compat --> linter
  linter --> prep
  prep --> config
  config --> utils
  utils --> core
  core --> client
  client --> trajectory
  trajectory --> verification
  verification --> metrics
  metrics --> artifacts
  artifacts --> srmods
  srmods --> su
  su --> model
  model --> modelbase
  modelbase --> agents
  agents --> data
  data --> harbor
  harbor --> rollouts
  rollouts --> rsplit
  rsplit --> testing
  testing --> orch
  orch --> sup
  sup --> eval
  eval --> env
  env --> cli
  cli --> docs
  docs --> owners
  owners --> remove
  classDef this fill:#fde68a,stroke:#b45309,stroke-width:2px,color:#1f2937
  classDef later fill:#f3f4f6,stroke:#9ca3af,color:#4b5563,stroke-dasharray: 4 3
  class agents this
  class owners,remove later
Loading

Where the package sits

flowchart TD
  cli["cli/"]
  eval["eval/"]
  environment["environment/"]
  orchestration["orchestration/"]
  rollouts["rollouts/"]
  data["data/"]
  agents["agents/"]
  model["models/"]
  server_runtime["server_runtime/"]
  artifacts["artifacts/"]
  metrics["metrics/"]
  verification["verification/"]
  trajectory["trajectory/"]
  sandbox["sandbox/"]
  client["client/"]
  telemetry["telemetry/"]
  core["core/"]
  utils["utils/"]
  config["config/"]
  cli --> eval
  cli --> environment
  cli --> orchestration
  eval --> rollouts
  environment --> data
  orchestration --> data
  rollouts --> data
  rollouts --> agents
  rollouts --> model
  data --> server_runtime
  data --> artifacts
  agents --> server_runtime
  model --> server_runtime
  server_runtime --> metrics
  server_runtime --> verification
  server_runtime --> trajectory
  artifacts --> trajectory
  artifacts --> utils
  metrics --> utils
  verification --> sandbox
  trajectory --> core
  sandbox --> client
  client --> telemetry
  client --> utils
  telemetry --> core
  core --> config
  utils --> config
  classDef this fill:#fde68a,stroke:#b45309,stroke-width:2px,color:#1f2937
  classDef done fill:#d1fae5,stroke:#047857,color:#1f2937
  classDef todo fill:#f3f4f6,stroke:#9ca3af,color:#4b5563
  class agents this
  class config,utils,core,telemetry,client,trajectory,verification,metrics,artifacts,server_runtime,model done
  class cli,eval,environment,orchestration,rollouts,data,sandbox todo
Loading

An arrow means "imports"; each package imports only packages below it. Yellow: created by this PR. Green: created by an earlier PR in the stack. Grey: still to come.

What moves

Whole modules, unchanged:

Old path New path
nemo_gym/base_responses_api_agent.py nemo_gym/agents/base_responses_api_agent.py
nemo_gym/agent_outcome.py nemo_gym/agents/agent_outcome.py
nemo_gym/agent_activation_types.py nemo_gym/agents/agent_activation_types.py
nemo_gym/skills.py nemo_gym/agents/skills.py
nemo_gym/sandbox/agent_tools.py nemo_gym/agents/agent_tools.py
nemo_gym/agent_utils/ nemo_gym/agents/utils/

How to review

One commit. Its changes:

  1. Renames, with no content changes.
  2. Everything else:
    • The moves are recorded in MOVED_MODULES in nemo_gym/_moved_paths.py.
    • In-repo references are rewritten by scripts/migrate_imports.py.
    • The contract's agents layer becomes the nemo_gym.agents package.
    • The contract's sandbox layer becomes the whole nemo_gym.sandbox package. With agent_tools gone, none of its submodules moves elsewhere, so its __init__.py is now checked too.
    • fern/docs.yml redirects the old library reference pages to the new ones.
    • agent_utils is recorded as a package, so its submodules, such as nemo_gym.agent_utils.sandbox_session, resolve too. Its harness and supervisor shell scripts move with it; the agents upload them from the host by path, so no sandbox imports them by name.
    • By hand:
      • the lazy exports in nemo_gym/__init__.py point at agents.base_responses_api_agent;
      • the harness guide names nemo_gym.agents.utils and the agents/utils/ files in its heading, cross-reference, shared-module table and diagram;
      • the add-benchmark skill's multi-line samples import from nemo_gym.agents and nemo_gym.server_runtime, because migrate_imports rewrites only one-line text imports.
  3. Tests: the agent server, outcome, activation, sandbox session, agent tools, and skills tests move into tests/unit_tests/agents/; test_sandbox_agent_tools.py becomes test_agent_tools.py with its module.

Public API

Every layout package that exists after this PR has a lazy public API in its __init__.py: the names code outside nemo_gym/ imports from it (servers and their tests, benchmarks, scripts, and code samples in docs and skills), plus every name it already exported, each served from the submodule that defines it on first use. Code outside nemo_gym/ and its tests (servers and their tests, benchmarks, scripts, and docs) imports those names from nemo_gym.<package>, so later moves inside a package leave it alone. Code inside nemo_gym/, and the library's tests under tests/, import from the module that defines each name, so the import-linter contract sees every dependency and a test reads what it patches. A patch target names the module the code under test looks the name up in. AGENTS.md states the rule.

Validation

  • pytest -n 16 tests/unit_tests, also with -W error for the moved paths. The only failures are environment-specific and unrelated to the move: this machine's hostname resolves to 127.0.1.1, which the sandbox host check refuses (the Harbor run summary tests and the benchmark listing test), and two Docker PTY tests.
  • lint-imports.
  • pre-commit run on the changed files.
  • Server and agent tests run in CI.

Rollout evidence: N/A. The modules move unchanged.

Compatibility

  • Old paths work until 0.10.0, with one warning each, and give the same module objects.
  • Logger names follow the module path, so these modules now log under nemo_gym.agents.*. A logging config set on nemo_gym still covers them.
  • Open branches: run python scripts/migrate_imports.py <paths>.

Issue

Part of #5166.

@ananthsub
ananthsub added this pull request to stack #5170 October 28, 2026 21:36
@copy-pr-bot

copy-pr-bot Bot commented Oct 28, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@ananthsub ananthsub added the feature New capabilities, enhancements, or enablement work label Oct 28, 2026
@ananthsub
ananthsub requested a review from yaoyu-33 October 28, 2026 21:36
@ananthsub ananthsub changed the title ananthsub/layout agents refactor(agents): move the agent server modules into nemo_gym/agents Oct 28, 2026
@ananthsub ananthsub added area:core Shared APIs, servers, telemetry, health, and registries needs-review PR is ready for code review and waiting on a reviewer labels Oct 28, 2026
@ananthsub
ananthsub marked this pull request as ready for review October 28, 2026 21:36
@ananthsub
ananthsub requested a review from a team as a code owner October 28, 2026 21:37
@nemo-automation-bot
nemo-automation-bot Bot requested review from a team October 28, 2026 21:37
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

@ananthsub
ananthsub force-pushed the ananthsub/layout-agents branch from 1f9f755 to 94444d8 Compare October 29, 2026 14:19
@ananthsub
ananthsub force-pushed the ananthsub/layout-agents branch from 94444d8 to ed65bc8 Compare October 29, 2026 15:14
@ananthsub
ananthsub force-pushed the ananthsub/layout-agents branch from ed65bc8 to b9b56aa Compare October 29, 2026 15:51
@ananthsub
ananthsub removed this pull request from stack #5170 October 29, 2026 15:55
@ananthsub
ananthsub added this pull request to stack #5319 October 29, 2026 15:56
@ananthsub
ananthsub force-pushed the ananthsub/layout-agents branch from b9b56aa to 335b7a1 Compare October 29, 2026 17:01

@yaoyu-33 yaoyu-33 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this one is clean. Verdict: ship. There are three small NOTEs inline and no blockers or risks.

What we checked at 5a77563ff2de

  • The move is mechanical.
    • All 19 renamed files are identical apart from their import lines, and both .sh scripts are byte-identical.
    • Across all 169 changed Python files, the AST differs only in imports and in the intended strings: lazy exports, MOVED_MODULES, scaffold templates, patch targets and logger names.
    • All 633 rewritten import bindings resolve to the same objects as before.
    • The 202 moved test IDs are unchanged.
  • Old import paths keep working.
    • Every old path returns the same module with one DeprecationWarning. That includes each nemo_gym.agent_utils.* submodule and nemo_gym.sandbox.agent_tools, also when imported as from nemo_gym.sandbox import agent_tools.
    • mock.patch through an old path still reaches the running code. Because these are whole-module moves, no patch target stops taking effect.
    • A base-tree pickle of AgentOutcome / SandboxCommand loads at head.
  • Public API, packaging and CI wiring.
    • nemo_gym.agents._PUBLIC_API (42 names) matches its TYPE_CHECKING block, and each name is used by non-test code outside the library.
    • lint-imports is KEPT.
    • fern/docs.yml redirects all 11 moved pages in both forms.
    • The session-routing CI filter is updated.
    • The wheel ships harness_runner.sh and process_supervisor.sh under agents/utils.
    • The runners that execute inside sandboxes import nothing this PR moves.
  • Tests pass.
    • The unit suite passes, and so do the server suites of every agent and environment server touched here, locally and in CI.
    • With this PR's moved-path DeprecationWarnings turned into errors, no test imports an old path.
  • CI.
    • The one red required job, Server suite (shard 5), fails only on responses_api_models/vllm_model test_cancelled_chat_keepalive_retains_uncertain_capture[False]. This PR does not touch that server, and the test passes 5/5 locally and on #5225 and #5229.
    • kilocode conformance is the known non-blocking model_error probe.
Stack-wide items rooted in other PRs (not for this PR)
  • #5167, scripts/migrate_imports.py
    • Text imports are rewritten to the package API only when they fit on one line, so multi-line samples in .md/.mdx keep the submodule path. The patterns.md note inline comes from this.
    • Earlier leftovers of the same kind: the server_runtime samples at patterns.md:24/56 and several fern tutorial pages from #5214 (e.g. single-step-environment.mdx:192).
  • #5189, AGENTS.md:194-197
    • The rule says server tests import from nemo_gym.<package>, and that a missing name gets added to _PUBLIC_API.
    • Server tests here import private names from submodules (_AgentSessionRecord, parse_cleanup_receipt, AGENT_SESSION_COOKIE_KEY, ...), and the facade test only enforces exported names.
    • The rule could say that server tests may import non-exported names from the defining module.
  • #5224 / #5225, nemo_gym/models/__init__.py
    • Two entries name modules that only re-export them:
      • CONTEXT_LENGTH_EXCEEDED_ERROR_CODE (:89) points at base_responses_api_model. It is defined in models/token_capture/records.py.
      • MAX_NUM_TRIES (:95) points at openai_utils. It is defined in client/http_client.py.
    • The root _LAZY_EXPORTS has a defining-module check, but no package facade test does. A check with an allowlist for intentional re-exports such as Body would catch these.

Comment thread pyproject.toml Resolved
yaoyu-33
yaoyu-33 previously approved these changes Nov 2, 2026

@yaoyu-33 yaoyu-33 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 5a77563ff2de. The team review found no blockers or risks; the remaining inline notes are optional and can be handled here or in a follow-up.

base_responses_api_agent.py, agent_outcome.py, agent_activation_types.py
and skills.py move as they are, sandbox/agent_tools.py becomes
agents/agent_tools.py, and the agent_utils package becomes agents/utils.

Record the six moves in MOVED_MODULES (agent_utils as a package),
rewrite in-repo references with scripts/migrate_imports.py, point the
lazy exports at agents.base_responses_api_agent, make the contract's
agents layer the nemo_gym.agents package, and redirect the old library
reference pages in fern/docs.yml.

Regenerate the public API of each layout package that exists here from
what non-test code outside nemo_gym imports, and point code outside
nemo_gym, tests included, at nemo_gym.<package>.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>

This branch was previously deployed

1 inactive deployment
public — aa13c064 Deployed Nov 2, 2026 by copy-pr-bot[bot] via release / finalize / notify #9338
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core Shared APIs, servers, telemetry, health, and registries feature New capabilities, enhancements, or enablement work ready-to-merge PR is approved, current, and waiting for final CI or merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants