[tylerpoon]

• I, the PR author, have personally reviewed every line of this PR.

What does this PR do?

Carry packed multimodal metadata through LLaVA, split/gather vision tokens for context parallelism, and handle padding and loss weighting consistently. Include model tests and single-writer dataloader checkpoint-state handling.

This is part of a review-split multimodal SFT series. Production run scripts, data YAMLs, checkpoint conversion tooling, and separate CI compatibility fixes are outside this series.

Dependencies and review scope

Stacked on #7249. Review the vision context-parallelism/metadata commits above that parent. The review base uses its bot-created mirror; retarget to main before merging.

Validation

• Replayed onto public main at 46fb90ca942a2950c58c5654b3e56877e88a3e35, preserving atomic signed and signed-off commits.
• The original combined feature stack had multi-node Super SFT validation before this migration; that is not an isolated-PR or rebased-head test result.
• GitHub CI is pending. No GPU tests have been rerun locally for these rebased heads.
• Local Black, isort, pylint, and Ruff checks passed for paths covered by the repository autoformat script (megatron/core and tests). Mypy is unavailable locally. Changed Python files pass syntax parsing; no CI configuration changes are included.

Kept as a draft pending CI and dependency integration.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[tylerpoon]

/ok to test ea7eee9

[tylerpoon]

/ok to test c4ca837