Repository navigation
ci: narrow harness conformance checks and label them non-blocking - #4628
Conversation
lafrancef
added
ci
|
/ok to test aa4553e9cdc362bb122b891fd5849fc59949052 |
@lafrancef, there was an error processing your request: See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/ |
|
/ok to test faa4553 |
|
/ok to test 62842c0 |
yaoyu-33
added
complexity:medium
github-actions
Bot
added
the
sla:triage-overdue
|
/review |
svcnvidia-nemo-ci
left a comment
svcnvidia-nemo-ci
left a comment
There was a problem hiding this comment.
HOLD — one RISK: the narrowed selector skips shared pytest infrastructure that the conformance validation job executes, allowing an affected change to receive a successful “No affected harnesses” check.
Reviewed all eight changed files and traced selection through probe validation and check publication. Static review only; tests were not executed.
Note: formal approval is not available for this request, so this review is posted as a comment only.
Review generated by codex
github-actions
Bot
removed
the
sla:triage-overdue
|
/ok to test 1258859 |
Signed-off-by: Frederic Lafrance <flafrance@nvidia.com>
Signed-off-by: Frederic Lafrance <flafrance@nvidia.com>
Signed-off-by: Frederic Lafrance <flafrance@nvidia.com>
1258859 to
2686aba
Compare
|
/ok to test 2686aba |
|
/review |
svcnvidia-nemo-ci
left a comment
svcnvidia-nemo-ci
left a comment
There was a problem hiding this comment.
SHIP — the prior RISK is fixed; no eligible findings remain in this follow-up.
Verified that select_harnesses selects every registered harness for all four shared pytest ancestor paths before applying the generic test skip rule. Routing regressions cover each path alone and combined with Pi in either order, while preserving unrelated-path skipping and Pi-only selection.
Inspected all eight changed-file hunks and traced the fix through probe validation and check publication. This was a follow-up review, not a fresh whole-PR assessment. The previously reviewed tree was unavailable, limiting independent comparison of inter-review changes. Static review only; tests were not executed.
Note: formal approval is not available for this request, so this review is posted as a comment only.
Review generated by codex
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/39418183513 |
What does this PR do?
Reduce noisy conformance runs by skipping unrelated servers, CLI/task ingestion, tests, and benchmarks. Registered adapter changes select that harness; shared probe dependencies and unknown paths select all four. Scheduled and manual runs cover the full registry.
Use the push's previous SHA and the merge group's base SHA to include every commit in those events. Name the workflow group and published check Harness conformance (non-blocking), with the reporting job shown as Harness conformance (non-blocking) / Publish check.
Follow-up to #4192 and #4489; no separate issue needed for this focused CI correction.
Validation
Replaying the reported PR file lists selects all four harnesses for #4609 and none for #4606, #3999, #3805, or #3673. Coverage also includes shared CLI compatibility helpers, mixed adapter/unrelated changes, and deleted or renamed paths across multiple commits.
Rollouts
N/A: CI routing and check names only.
Compatibility and benchmark impact
The published check name changes. Conformance failures still produce failed checks independently of
Nemo_CICD_Test; benchmark scoring is unaffected.