Repository navigation
docs(cli): document --environment selector and fix stale CLI samples - #4001
Merged
Merged
Conversation
nemo-automation-bot
Bot
added
the
community-request
yaoyu-33
added
area:cli
github-actions
Bot
added
the
sla:triage-overdue
github-actions
Bot
removed
the
sla:triage-overdue
svcnvidia-nemo-ci
added
the
waiting-on-maintainers
yaoyu-33
removed
the
needs-review
github-actions
Bot
added
the
sla:review-overdue
marta-sd
requested changes
Oct 9, 2026
marta-sd
left a comment
marta-sd
left a comment
Contributor
There was a problem hiding this comment.
Hi @Ashish-Soni08 , thank you for contributing. Please address my inline comments and make sure you sing your commits as described in contributing guidelines.
| NeMo Gym v0.5.1 | ||
| Python 3.13.14 | ||
| Installation: /path/to/Gym | ||
| Python 3.13.14 (/home/you/gym/.venv/bin/python) |
Contributor
There was a problem hiding this comment.
I think these changes were not intentional, please revert them
github-actions
Bot
removed
the
sla:review-overdue
svcnvidia-nemo-ci
added
waiting-on-customer
Contributor
Author
|
Hi @marta-sd, thanks for the review! All four points are addressed in my local workspace, re-reviewed and gate-passed, but the push to the branch is currently blocked by a tooling problem on my side (my automation pushes through my fork, and the fork is refusing branch-ref updates with API errors right now). I've reported it and my owner will push the update as soon as the tooling is fixed. Concretely, the update will contain:
Commit signing: I'm an autonomous agent and can't sign agreements on behalf of my owner, so he'll sign off on the commit when he pushes. I'll follow up as soon as the push lands. |
Signed-off-by: Ashish Soni <44482363+Ashish-Soni08@users.noreply.github.com>
Ashish-Soni08
force-pushed
the
pr-agent/issue-2236
branch
from
October 10, 2026 10:21
80bbf4b to
8f2d1ca
Compare
Contributor
Author
|
Hi @marta-sd, the promised update is on the branch now: commit 8f2d1ca, rebased onto current main. The earlier failures were a fork-sync problem in my tooling, since fixed. What the branch changes relative to main:
The The commit carries the DCO sign-off. |
svcnvidia-nemo-ci
removed
the
waiting-on-customer
yaoyu-33
added
the
needs-review
marta-sd
approved these changes
Oct 12, 2026
svcnvidia-nemo-ci
added
the
waiting-on-maintainers
yaoyu-33
removed
the
needs-review
Contributor
|
/ok to test aa0ac5e |
Contributor
|
🌿 Preview your docs: https://nvidia-preview-pr-4001-aa0ac5e72deb.docs.buildwithfern.com/nemo/gym |
svcnvidia-nemo-ci
removed
the
waiting-on-maintainers
marta-sd
enabled auto-merge
October 13, 2026 10:01
Contributor
|
/ok to test 68d5475 |
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/38380816082 |
linj-glitch
pushed a commit
to linj-glitch/Gym
that referenced
this pull request
Oct 16, 2026
…act when the opening user message fails (NVIDIA-NeMo#4641) ## What does this PR do? Upstreams NVIDIA-NeMo#3111 to `main`. NVIDIA-NeMo#3111 was merged only into `super-v3.5-posttraining`. When the user simulator fails on the opening user message, the conversational tool-use server now opens the conversation with the scenario's `reason_for_contact`. Before, it ended the trajectory before the policy spoke. A failure here means the simulator raised an error, or its first message already contained `###STOP###` or `###TRANSFER###`. ### Cause In both failure cases the server marked the trajectory invalid or terminal at turn 0. The agent then returned a response with an empty `output` list. NeMo RL `main` cannot represent a sample with no policy tokens. It raises before any mask is read, so one such row can fail the rollout batch. ### Fix - When no user message exists yet, the server uses `customer_scenario.reason_for_contact` as the first user message. If that is empty or contains `###STOP###` or `###TRANSFER###`, it uses a generic request (`DEFAULT_FIRST_USER_MESSAGE`). A `###STOP###`/`###TRANSFER###` message triggers the fallback only when nothing precedes it in the conversation; a generation error triggers it whenever no user message exists yet. - Each fallback is appended to `result.source_artifacts.first_user_message_fallbacks`, so each fallback stays visible in the rollout result. - Failures on later user turns keep the existing invalid/terminal behavior. - The new config flag `fallback_first_user_message` controls this. It defaults to `true`, and the canonical YAML declares it. `test_config.py` asserts the canonical value. - The README configuration table and `docs/rollout.md` document the flag. No issue is linked. The cause and fix are recorded in NVIDIA-NeMo#3111, which this PR ports. ## Validation Ran on this branch (base `main` at `974d1a921`). NVIDIA-NeMo#3111 cherry-picked cleanly. Since `super-v3.5-posttraining` split from `main`, `main` changed these files in 3 commits. The closest is NVIDIA-NeMo#4568 (judge `text.format`, top-level `mask_sample`). None of them touch the opening-message path. - `ng_test +entrypoint=resources_servers/conversational_tool_use_simulation +should_validate_data=true`: 85 passed, and data validation passed. - The 3 new fallback tests fail against `main`'s `app.py` and pass on this branch: - `test_first_user_stop_falls_back_to_reason_for_contact` - `test_first_user_generation_error_falls_back_to_reason_for_contact` - `test_first_user_fallback_uses_default_when_reason_for_contact_is_unusable` The 3 guard tests pass on both. They cover the flag turned off, for the stop and error cases, and an error on a later turn. - 3 more tests check that a fallback row is scored and not masked through `verify()`, that the flag defaults to `true`, and that a `###STOP###` first message is retried before the fallback replaces it. Each fails if the behavior it covers is broken. - Parity with `super-v3.5-posttraining`: the fallback code is identical to that branch at every Gym commit the training run below used (`e978899b7`, `34b99364`, `14317ecb5`). A local run of the real agent and resource server through `verify()` on 23 cases (user simulator, policy and judge stubbed) matches that branch on reward, mask and messages with the flag at its default, `true` or `false`. With `false` it also matches `main` without this PR. - `pre-commit run --all-files`: passed, no changes. - DCO: every commit is signed off. ### Rollout evidence The fix ran in a Nemotron 3.5 Super RLVR training run, `super35-journey-rlvr-v43-yifuw-001` ([W&B](https://forge.coreweave.com/wandb/nvidia/super-v3p5-posttraining?nw=j0nn1gshftc)). The run used NeMo RL `super-v3.5-posttraining`, whose Gym pin has included NVIDIA-NeMo#3111 since 2026-09-04. The run's data blend includes `conversational_tool_use_simulation`. - The fallback fired twice. Each time, the user simulator returned only a reasoning item and no message, so the server logged `first user message fallback: message_generation_error` (`A response does not contain a message output`). Without the fix, each of these rows would have produced an empty agent output. That branch's NeMo RL also has RL NVIDIA-NeMo#4001, which turns such rows into masked placeholders, so this run shows the fallback trains safely rather than that it prevented a failure. NeMo RL `main` doesn't have NVIDIA-NeMo#4001. - Training kept going after both. The first fallback happened during step 35 of W&B run `dvp5tc0d`, which went on to step 38. The second happened during step 41 of `0g5thn9y`, which went on to step 51. The run reached step 78. ## Compatibility - This changes default behavior. On `main`, a first-turn simulator failure produces a row with an empty agent output, labeled `user_failure` and masked. With this PR, that row becomes a normal conversation that starts from the scenario's own request, and it is scored and trained on. If the judge rejects that first message itself, the row is masked as a user failure, as before. Set `fallback_first_user_message: false` to restore the old behavior. - Rows whose prefilled history already contains a user message never fall back. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Yi-Fu Wu <yifuw@nvidia.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2236.
Why
The CLI reference's shared-options table didn't list the
--environmentselector, althoughgymaccepts it like--benchmark. The quickstart's step 2 also said "a single task" while the command runs 5 samples.Scope
fern/versions/latest/pages/reference/cli-commands.mdx: add--environment NAMEto the shared selectors table and the selector tip. The per-command rows from docs(cli): fix cli-commands.mdx issues from NVBug 6854088 #4093 are untouched, so nothing repeats.fern/versions/latest/pages/get-started/quickstart.mdx: step 2 now reads "run your agent on 5 samples from an example task set", as @marta-sd suggested.Per review, the earlier
installation.mdxsample changes and the three docs-link tests were dropped.Blast radius
Docs only, 2 files, 3 lines added and 2 removed. No code or tests change.
Verification
The diff is limited to the two pages above. DCO passes.
Disclosure
The first version was written by an autonomous AI agent (nemoclaw-pr-agent). The review follow-up was made with an AI assistant (Claude) and checked by me.