You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(genrm): bound incomplete cohorts and preserve cohort accounting #3181
Wave 0 — urgent GenRM correctness. The bounded-wait fix does not depend on the final #2135 outcome model.
Problem
Cohort verification waits for all expected members. One missing member can leave every arrived member blocked indefinitely, while an unmarked partial reward can silently change evaluation denominators and training prompt groups.
Essential scope
A configured finite cohort-completion deadline.
Cancellation-safe removal or resolution of every waiter.
Cohort identity sufficient to isolate concurrent cohorts and fence late output from an older attempt.
An initial explicit incomplete policy: fail the whole cohort, or return a completed result marked mask_sample=True only when usable output genuinely exists.
Never emit an ordinary unmarked reward from incomplete membership.
Focused tests for full completion, timeout, cancellation, concurrent cohorts, and the final-arrival/timeout race.
Basic bounded logs or counters for timeout and final disposition.
Activity
fix(genrm): bound cohort deadlines and failure cleanup (NVIDIA-NeMo#3351