[Silas-Zeng]

What does this PR do?

#7608 adds typed payload boundaries for non-interleaved PP. VPP still rejects the communicator, and a single payload queue cannot distinguish virtual chunks across the physical last-to-first wrap.

This patch adds per-chunk payload factories, plans, cursors, send/receive queues, and source/destination chunk identities. The interleaved schedule passes chunk IDs through warmup, steady state, and cooldown and checks that every planned payload is consumed.

Issue tracking

Related to #7565. Depends on #7608.

Scope

Supported path: eager training with non-overlapped PP and TP=CP=1. Recompute, CUDA graphs, packed sequences, and overlapping P2P remain rejected at communicator entry.

Tests

• Baseline: unmodified Add prepared PP boundaries for shared cross-layer state #7608 reproduces the VPP constructor failure for virtual_pipeline_model_parallel_size=2.
• 7 focused payload and VPP bookkeeping tests.
• Two-rank CPU/Gloo transport with distinct chunk schemas and the cyclic VPP wrap route.
• Two-process single-GPU interleaved forward/backward schedule with CUDA tensors on an RTX 3050 Ti.
• Single-GPU CUDA transport smoke on an RTX 3050 Ti.
• Black 26.3.0, isort 5.13.2, Ruff, compileall, and git diff --check.

The tested environment has one physical GPU; multi-GPU NCCL and full-model convergence still require the upstream CI or a multi-GPU run.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.