Last step checkpoint save for GRPO creates a checkpoint that doesn't have val_reward in training_info.json

[soluwalana]

Describe the bug

Unless the max_steps % val_period == 0 the val_metrics is not calculated for the last checkpoint so when it is saved it doesn't have a val_reward field. in GRPO

This causes the code in get_best_checkpoint_path in utils/checkpoint.py to fail with the following error:

Traceback (most recent call last):
  File "/app/services/customizer/src/training/nemo/config/../rl/run_grpo_penguin.py", line 277, in <module>
    main()
  File "/app/services/customizer/src/training/nemo/config/../rl/run_grpo_penguin.py", line 245, in main
    best_checkpoint = checkpointer.get_best_checkpoint_path()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/src/nemo-rl/nemo_rl/utils/checkpoint.py", line 246, in get_best_checkpoint_path
    checkpoint_history.sort(
  File "/app/src/nemo-rl/nemo_rl/utils/checkpoint.py", line 247, in <lambda>
    key=lambda x: x[2][self.metric_name], reverse=self.higher_is_better
                  ~~~~^^^^^^^^^^^^^^^^^^
KeyError: 'val_reward'

Recommend changing this code in the grpo.py algorithm where determining whether to run validations:

if val_period > 0 and (step + 1) % val_period == 0 :

to

if (val_period > 0 and (step + 1) % val_period == 0) or is_last_step:

https://github.com/NVIDIA-NeMo/RL/blob/main/nemo_rl/algorithms/grpo.py#L1937

[terrykong]

@soluwalana i think it might be better to make get_best_checkpoint_path more robust and add a flag to validate at the end so that the user has control over this since the validation could take a while.

could you provide feedback on #1863 to see if this would help resolve your issue?

[qiaochuz-nv]

QA RCCA Analysis

RCCA Status: Done
RCCA Category: covered
Test Action: none

Fix PR: #1863
Root Cause: fix PR includes upstream test coverage.
Trigger Config: Last step checkpoint save for GRPO creates a checkpoint that doesn't have val_reward in training_info.json

Coverage Assessment:

Area	Result
Existing Test	Upstream fix PR test coverage
Coverage Status	COVERED
Gap	N/A
Corrective Action	No nmfw_tests addition required; fix PR includes upstream test coverage or equivalent regression coverage.

Conclusion: RCCA classification completed for this closed NeMo-RL GitHub issue.