Repository navigation
Thread socket argument through MInf API - #6513
Merged
Merged
Conversation
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
kvareddy
approved these changes
Aug 13, 2026
YangFei1990
approved these changes
Aug 13, 2026
shanmugamr1992
approved these changes
Aug 13, 2026
svcnvidia-nemo-ci
added
Approved
tdene
enabled auto-merge
August 13, 2026 21:28
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/31751169913 |
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/31752551299 |
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/31752940303 |
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/31755641698 |
ksivaman
pushed a commit
to ksivaman/Megatron-LM
that referenced
this pull request
Sep 1, 2026
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
nemo-autobot-origin Bot
pushed a commit
to svcnemo-autobot/Megatron-LM
that referenced
this pull request
Oct 15, 2026
Repair commit for the 2026-10-13 main-to-dev sync merge 4ade8e7. - inference: restore main NVIDIA#5301 pre-bound socket support (the sock parameter) in start_text_gen_server. The slice's NVIDIA#6513 makes MegatronLLM.serve() and MegatronAsyncLLM.serve() pass sock=, but dev's server lacked NVIDIA#5301, so serve() raised TypeError on the primary rank. Add tests that bind both serve() calls to the real signature and cover the pre-bound socket path. - inference: restore main NVIDIA#5348/NVIDIA#6584 mamba-cache scratch reservation and budget-too-small / PP-limiting-stage errors in DynamicInferenceContext (the dev warn-and-disable path was an artifact of revert NVIDIA#7283), and restore test_mamba_cache_budget_too_small_raises. - megatron_fsdp experimental: keep the symmetric-memory all-gather buffer rendezvous working on the dev torch build, which lacks symm_mem.is_symm_mem_tensor. - functional moe_perf: restore the per-iteration CUDA RNG tracker state reset from NVIDIA#7094. - unit test doubles: align with merged APIs (RecordingCheckpoint retain_input_tensors, paged-stash config cuda_graph fields, profile_ranks in test_training, contextmanager import and the dev pipeline_p2p_fixed_shape config field in test_schedules). - unit test isolation: reset the global CUDA graph record after the dev TestMHCWithCudaGraph local-impl tests and before main NVIDIA#6432's TestTransformerLayerCudaGraphManagers, so its create_cudagraphs() does not capture stale mHC layers recorded under the cudagraphable RNG tracker. - unit test teardown: Utils.destroy_model_parallel resets the TE FP8 global state and the HybridEP buffer that cache process groups, and retires the groups parallel_state created; a new autouse conftest fixture destroys them once the test finishes. They are not destroyed inside destroy_model_parallel because a test may re-initialize and keep using a group it captured earlier (test_upcycling_TE reuses its ProcessGroupCollection). Before this, each test leaked its NCCL communicators. The tests/unit_tests/transformer bucket then crossed vm.max_map_count (65530) at test_vision_cuda_graphs, on both this branch and dev c75a269. Before destroying the retired groups, the fixture clears DTensor's per-thread sharding-propagation caches (Python and native). They key on value-equal DeviceMeshes, so a later test that builds a mesh with the same layout would otherwise get placements on the earlier mesh, whose group is destroyed (test_gpt_hybrid_interop fsdp4-dense). - rl unit tests: a bucket-scoped autouse conftest fixture runs gc.collect() before the retired groups are destroyed. test_get_logprobs_cuda_graphs leaves CUDA graphs that captured those communicators reachable only through reference cycles, and ncclCommDestroy waits for them to be freed, so the teardown hung. Collecting suite-wide instead changed object lifetimes in other buckets, so the collection stays in tests/unit_tests/rl. Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
nemo-autobot-origin Bot
pushed a commit
to svcnemo-autobot/Megatron-LM
that referenced
this pull request
Oct 16, 2026
Repair commit for the 2026-10-13 main-to-dev sync merge 1eaec47. - inference: restore main NVIDIA#5301 pre-bound socket support (the sock parameter) in start_text_gen_server. The slice's NVIDIA#6513 makes MegatronLLM.serve() and MegatronAsyncLLM.serve() pass sock=, but dev's server lacked NVIDIA#5301, so serve() raised TypeError on the primary rank. Add tests that bind both serve() calls to the real signature and cover the pre-bound socket path. - inference: restore main NVIDIA#5348/NVIDIA#6584 mamba-cache scratch reservation and budget-too-small / PP-limiting-stage errors in DynamicInferenceContext (the dev warn-and-disable path was an artifact of revert NVIDIA#7283), and restore test_mamba_cache_budget_too_small_raises. - megatron_fsdp experimental: keep the symmetric-memory all-gather buffer rendezvous working on the dev torch build, which lacks symm_mem.is_symm_mem_tensor. - functional moe_perf: restore the per-iteration CUDA RNG tracker state reset from NVIDIA#7094. - unit test doubles: align with merged APIs (RecordingCheckpoint retain_input_tensors, paged-stash config cuda_graph fields, profile_ranks in test_training, contextmanager import and the dev pipeline_p2p_fixed_shape config field in test_schedules). - unit test isolation: reset the global CUDA graph record after the dev TestMHCWithCudaGraph local-impl tests and before main NVIDIA#6432's TestTransformerLayerCudaGraphManagers, so its create_cudagraphs() does not capture stale mHC layers recorded under the cudagraphable RNG tracker. - unit test teardown: Utils.destroy_model_parallel resets the TE FP8 global state and the HybridEP buffer that cache process groups, and retires the groups parallel_state created; a new autouse conftest fixture destroys them once the test finishes. They are not destroyed inside destroy_model_parallel because a test may re-initialize and keep using a group it captured earlier (test_upcycling_TE reuses its ProcessGroupCollection). Before this, each test leaked its NCCL communicators. The tests/unit_tests/transformer bucket then crossed vm.max_map_count (65530) at test_vision_cuda_graphs, on both this branch and dev c75a269. Before destroying the retired groups, the fixture clears DTensor's per-thread sharding-propagation caches (Python and native). They key on value-equal DeviceMeshes, so a later test that builds a mesh with the same layout would otherwise get placements on the earlier mesh, whose group is destroyed (test_gpt_hybrid_interop fsdp4-dense). - rl unit tests: a bucket-scoped autouse conftest fixture runs gc.collect() before the retired groups are destroyed. test_get_logprobs_cuda_graphs leaves CUDA graphs that captured those communicators reachable only through reference cycles, and ncclCommDestroy waits for them to be freed, so the teardown hung. Collecting suite-wide instead changed object lifetimes in other buckets, so the collection stays in tests/unit_tests/rl. Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Issue tracking
For PRs from open-source community contributors:
Linked issue:
Contribution process
Pre-checks
Code review
Feel free to message or comment @NVIDIA/mcore-oncall to help accelerate your merge into main. The less complex your PR is, the faster it will be approved and merged!
All PRs start as draft. If you open a non-draft PR, it will be automatically converted to draft.
Step 1: Mark PR as "Ready for Review"
.github/CODEOWNERS.Final Review might get declined if these requirements are not fulfilled.
Step 2: Final Review
For PRs that change
megatron/core, once all expert reviewers have approved, theFinal Reviewlabel is applied automatically and final reviewers are assigned.For PRs outside
megatron/core, this step is skipped.Step 3: Approved
Once all required reviewers have approved, the
Approvedlabel is applied automatically.Merge
Any member of mcore-engineers will be able to merge your PR.