Repository navigation
Conversation
Signed-off-by: Tyler Poon <tpoon@nvidia.com>
Signed-off-by: Tyler Poon <tpoon@nvidia.com>
Signed-off-by: Tyler Poon <tpoon@nvidia.com>
Signed-off-by: Tyler Poon <tpoon@nvidia.com>
Signed-off-by: Tyler Poon <tpoon@nvidia.com>
1 task done
Signed-off-by: Tyler Poon <tpoon@nvidia.com>
Signed-off-by: Tyler Poon <tpoon@nvidia.com>
matthieule
approved these changes
Sep 15, 2026
tylerpoon
marked this pull request as ready for review
September 16, 2026 17:04
Signed-off-by: Tyler Poon <tpoon@nvidia.com>
Phlip79
reviewed
Sep 18, 2026
Phlip79
reviewed
Sep 18, 2026
Phlip79
reviewed
Sep 18, 2026
Phlip79
reviewed
Sep 18, 2026
Phlip79
reviewed
Sep 18, 2026
Signed-off-by: Tyler Poon <tpoon@nvidia.com>
Signed-off-by: Tyler Poon <tpoon@nvidia.com>
trintamaki
reviewed
Sep 18, 2026
trintamaki
reviewed
Sep 18, 2026
Keep per-replica CP partition diagnostics while removing the global TP rank lookup from the splitter. Simplify the unreachable video frame-count fallback. Signed-off-by: Tyler Poon <tpoon@nvidia.com>
Signed-off-by: Tyler Poon <tpoon@nvidia.com>
jaredcasper
approved these changes
Sep 18, 2026
tylerpoon
enabled auto-merge
September 18, 2026 20:08
Phlip79
approved these changes
Sep 18, 2026
Member
|
/ok to test 76dfaae |
svcnvidia-nemo-ci
added
Approved
trintamaki
approved these changes
Sep 18, 2026
Collaborator
|
Formal |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/35415930608 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/35416653178 |
tylerpoon
deleted the
tpoon/github-vlm-sft-03-vision-context-parallel
branch
September 19, 2026 04:54
Connor-XY
added a commit
to Connor-XY/Megatron-LM
that referenced
this pull request
Oct 7, 2026
test_forward_video_accounts_for_native_merging (added upstream in NVIDIA#7315) builds a bare LLaVAModel and patched the global context-parallel size and rank. LLaVAModel.forward now reads self.pg_collection.cp and passes it to the dynamic-resolution CP helpers, which size the split from that group, so the test failed with "'LLaVAModel' object has no attribute 'pg_collection'". Give the model a collection with a sentinel CP group, patch the group-based size and rank lookups in context_parallel instead, and check that the gather receives the same group. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Yan Xu <yxu1@nvidia.com>
Connor-XY
added a commit
to Connor-XY/Megatron-LM
that referenced
this pull request
Oct 8, 2026
test_forward_video_accounts_for_native_merging (added upstream in NVIDIA#7315) builds a bare LLaVAModel and patched the global context-parallel size and rank. LLaVAModel.forward now reads self.pg_collection.cp and passes it to the dynamic-resolution CP helpers, which size the split from that group, so the test failed with "'LLaVAModel' object has no attribute 'pg_collection'". Give the model a collection with a sentinel CP group, patch the group-based size and rank lookups in context_parallel instead, and check that the gather receives the same group. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Yan Xu <yxu1@nvidia.com>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Carry packed multimodal metadata through LLaVA, split/gather vision tokens for context parallelism, and handle padding and loss weighting consistently. Include model tests and single-writer dataloader checkpoint-state handling.
Replaces #7250, which was automatically closed when its bot-managed base branch was deleted after #7249 merged. The original discussion remains in #7250. This replacement reuses the original source branch and commits without code changes.
Production run scripts, data YAMLs, checkpoint conversion tooling, and separate CI compatibility fixes are outside this series.
Dependencies and review scope
#7249 has merged. This PR targets
mainand will remain based onmain, not a bot mirror. Merge this PR before #7316 (the language-only checkpoint-initialization replacement for #7251). That dependent PR also targetsmain; the author will manage merge order.The retained branch predates the squash merge of #7249, so its commit history/review diff may still include inherited vision-utility changes until refreshed against current main. Do not interpret those inherited changes as a new feature request.
Validation
c4ca837e4ffaa0705e95778024ef4564d25f11c1.