Skip to content

fix: correct QARL nightly quantization and HybridEP topology - #4548

Merged
yuki-97 merged 3 commits into
r0.8.0from
mxin/fix-qarl-real-quant-nightly
Oct 10, 2026
Merged

fix: correct QARL nightly quantization and HybridEP topology#4548
yuki-97 merged 3 commits into
r0.8.0from
mxin/fix-qarl-real-quant-nightly

Conversation

@mxinO

@mxinO mxinO commented Oct 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Targets r0.8.0; cherry-pick-to-main forwards the fix after the release merge.

Summary

  • Align Nano3 W4A16 policy and rollout quantization with the existing experts-only recipe.
  • Keep Qwen3 GB200 HybridEP ranks in one NVLink domain; use node-local topology for the H100 4n8g recipe.
  • Remove the arbitrary H100 reward ceiling while retaining the reward floor and other checks.
  • Relax Qwen3 W4A4's step-2 JS bound from <0.007 to <0.008 for modest run-to-run variation.

Five recipe/nightly files only. No Python, dependency, or submodule changes; no unrelated main commits.

Validation

  • Clean release patch: 115 existing topology/configuration tests, all three recipe minimization checks, repository Ruff checks, shell syntax, and whitespace checks pass.
  • Earlier snapshots: Nano3 W4A16 and Qwen3 W4A4 real-quant nightlies passed two steps. H100 training completed with reward 0.625; its saved metrics pass all 14 retained checks after removing the ceiling.
  • Earlier Qwen3 two-step metrics pass all five checks, including JS 0.00675325.
  • Aggregate source review has no remaining findings. Source-matched CI:L1 has been requested on the unchanged release head; results are pending. Lfast reused main's image and rebuilt TensorRT-LLM in every test job.

@copy-pr-bot

copy-pr-bot Bot commented Oct 6, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@mxinO
mxinO marked this pull request as ready for review October 7, 2026 17:51
@copy-pr-bot

copy-pr-bot Bot commented Oct 7, 2026

Copy link
Copy Markdown

Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@mxinO mxinO added the CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version) label Oct 7, 2026
@mxinO mxinO mentioned this pull request Oct 9, 2026
Merged
@mxinO mxinO added the r0.8.0 Auto-cherrypick to release branch. Apply before merge; cherrypick happens after merge. label Oct 9, 2026
@mxinO
mxinO requested a review from yuki-97 October 9, 2026 04:03
@mxinO

mxinO commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test

@mxinO

mxinO commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 2970b7d

yuki-97
yuki-97 reviewed Oct 9, 2026
View reviewed changes

@yuki-97 yuki-97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge blockers — please target the r0.8.0 branch instead of main and add the cherry-pick-to-main label, since this is a fix PR for the v0.8 release. Please also remove the r0.8.0 label, since the flow is now reversed: the fix lands on r0.8.0 first and is then cherry-picked back to main.

Comment thread nemo_rl/algorithms/grpo.py Resolved
@mxinO mxinO added cherry-pick-to-main Opt in: cherry-pick this PR's commit from the release branch back into main and removed r0.8.0 Auto-cherrypick to release branch. Apply before merge; cherrypick happens after merge. labels Oct 9, 2026
@mxinO
mxinO changed the base branch from main to r0.8.0 October 9, 2026 17:06
@mxinO
fix: correct QARL nightly quantization and HybridEP topology
3dd341f 
Signed-off-by: Meng Xin <164952785+mxinO@users.noreply.github.com>
@mxinO
mxinO force-pushed the mxin/fix-qarl-real-quant-nightly branch from 2970b7d to 3dd341f Compare October 9, 2026 19:37
@mxinO

mxinO commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 3dd341f

@mxinO
fix: inherit the equivalent Nano3 rollout quantization recipe
575cd8b 
Signed-off-by: Meng Xin <164952785+mxinO@users.noreply.github.com>
@mxinO

mxinO commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 575cd8b

@mxinO mxinO added CI:L1 Run doctests, unit tests, and functional tests and removed CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version) labels Oct 10, 2026
@yuki-97 yuki-97 added CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version) and removed CI:L1 Run doctests, unit tests, and functional tests labels Oct 10, 2026
@yuki-97
yuki-97 enabled auto-merge (squash) October 10, 2026 07:16
@yuki-97

yuki-97 commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

/ok to test bff1bee

@yuki-97
yuki-97 merged commit dc903f0 into r0.8.0 Oct 10, 2026
165 of 167 checks passed
@yuki-97
yuki-97 deleted the mxin/fix-qarl-real-quant-nightly branch October 10, 2026 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yuki-97 yuki-97 yuki-97 approved these changes
@sharonyu-115 sharonyu-115 Awaiting requested review from sharonyu-115
@terrykong terrykong Awaiting requested review from terrykong

Assignees

No one assigned

Labels

cherry-pick-to-main Opt in: cherry-pick this PR's commit from the release branch back into main CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mxinO @yuki-97