[trainer,reward] fix: graceful shutdown on Ctrl+C to prevent zombie GPU processes

[Jayce-Ping]

Summary

• Add RewardBuffer.shutdown() for non-blocking executor termination on interrupt
• Add BaseTrainer.cleanup() that calls shutdown on reward buffers
• Wrap trainer.start() with KeyboardInterrupt handler in train.py, using os._exit(0) to bypass NCCL heartbeat noise
• Catch SIGINT exit codes in cli.py for clean subprocess teardown

Context

When users press Ctrl+C during multi-GPU training, worker processes could become orphaned holding GPU memory. Root cause: the RewardBuffer's ThreadPoolExecutor (holding GPU references via CUDA streams) was never shut down on interrupt, and NCCL's C++ heartbeat threads kept running during Python's normal shutdown sequence.

Test plan

• Launch multi-GPU training with ff-train, press Ctrl+C during evaluation/sampling
• Verify nvidia-smi shows no zombie processes after exit
• Verify log shows "Training interrupted by user" then clean exit
• Verify normal training completion is unaffected

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR improves Ctrl+C handling during (multi-GPU) training to avoid orphaned worker processes that keep GPU memory allocated, by explicitly shutting down async reward executors and exiting cleanly when interrupted.

Changes:

• Add RewardBuffer.shutdown() for fast, non-reinitializing teardown of async reward executors.
• Add BaseTrainer.cleanup() to shut down training/eval reward buffers on interrupt.
• Wrap trainer.start() in train.py with KeyboardInterrupt handling and force-exit to avoid noisy/hanging shutdown paths; handle SIGINT-like subprocess exit codes in cli.py.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/flow_factory/trainers/abc.py	Adds cleanup() to shut down reward buffers during interrupt-driven teardown.
src/flow_factory/train.py	Catches KeyboardInterrupt, calls trainer cleanup, and exits immediately to avoid lingering shutdown noise.
src/flow_factory/rewards/reward_processor.py	Introduces RewardBuffer.shutdown() to terminate the underlying executor without resetting state.
src/flow_factory/cli.py	Treats SIGINT-style subprocess exits as a clean interrupt and exits with code 0.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Adds explicit interrupt/teardown hooks to ensure multi-GPU training exits cleanly on Ctrl+C without leaving orphaned processes holding GPU memory, primarily by shutting down async reward executors and force-exiting to avoid noisy/disruptive NCCL shutdown behavior.

Changes:

• Add RewardBuffer.shutdown() for fast, non-blocking threadpool teardown on interrupt.
• Add BaseTrainer.cleanup() to trigger reward-buffer shutdowns during interrupt handling.
• Handle KeyboardInterrupt in train.py with cleanup + os._exit(0), and treat SIGINT-like subprocess exits as clean interrupts in cli.py.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
src/flow_factory/trainers/abc.py	Introduces BaseTrainer.cleanup() that shuts down training/eval reward buffers without blocking.
src/flow_factory/train.py	Wraps trainer.start() in a KeyboardInterrupt handler, calls trainer.cleanup(), then os._exit(0).
src/flow_factory/rewards/reward_processor.py	Adds RewardBuffer.shutdown() to terminate the async ThreadPoolExecutor without reinitialization.
src/flow_factory/cli.py	Treats SIGINT-style subprocess termination as a clean interrupt to avoid noisy tracebacks and ensure exit(0).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

This PR improves Ctrl+C handling during training to avoid orphaned/zombie GPU processes by introducing explicit non-blocking shutdown hooks for async reward workers and handling SIGINT exit paths more deliberately in both the training entrypoint and CLI launcher.

Changes:

• Add RewardBuffer.shutdown() to stop the reward async executor without waiting/reinitializing.
• Add BaseTrainer.cleanup() to trigger reward-buffer shutdown on interruption.
• Catch KeyboardInterrupt in train.py and force-exit, plus treat SIGINT returns as clean exits in cli.py.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/flow_factory/trainers/abc.py	Adds BaseTrainer.cleanup() to shutdown reward buffers on interrupt.
src/flow_factory/train.py	Wraps training start in KeyboardInterrupt handling and force-exits after cleanup.
src/flow_factory/rewards/reward_processor.py	Adds RewardBuffer.shutdown() for non-blocking executor termination.
src/flow_factory/cli.py	Treats SIGINT/KeyboardInterrupt from the training subprocess as a clean exit.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Improves Ctrl+C (SIGINT) handling during training to avoid orphaned processes holding GPU memory, primarily by proactively shutting down async reward worker executors and exiting quickly.

Changes:

• Add a RewardBuffer.shutdown() method for fast, non-blocking executor teardown on interruption.
• Introduce BaseTrainer.cleanup() to shut down reward buffers’ async executors.
• Add SIGINT/KeyboardInterrupt handling in train.py (including forced exit) and in cli.py to treat Ctrl+C as a clean termination.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/flow_factory/trainers/abc.py	Adds BaseTrainer.cleanup() to shut down reward buffers’ async executors.
src/flow_factory/train.py	Wraps trainer.start() with KeyboardInterrupt handling and forces immediate process exit.
src/flow_factory/rewards/reward_processor.py	Adds RewardBuffer.shutdown() for final teardown of the thread pool executor.
src/flow_factory/cli.py	Treats SIGINT-like subprocess termination and KeyboardInterrupt as clean exits.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.