[Connor-XY]

This PR adds the Megatron Core APIs that later steps of the parallel_state deprecation (#6307) build on. Callers get a builder that returns the standard grid's ProcessGroupCollection without writing globals. They also get helpers that validate a collection or fall back to the globals with a dedicated warning. The global memory buffer no longer needs initialize_model_parallel(). Maintainers get shared test helpers that fail a test on any global grid read. It also makes DDP's fallback expert data-parallel group a collective call. It consolidates six draft PRs unchanged. Five are one commit each, so please review commit by commit; #8820 needs no commit of its own (see below). It builds on #6259 (#8017 and #8039 use its Gloo fields) and #6293 (#8049 builds on its fallback defaults).

Review

The unit's own commits: Connor-XY/Megatron-LM@7c8c403...eb768e2

The full PR diff also includes #7487, #6259, #6293 and #6303 until they merge.

Members

• Refresh the process-group read allowlist #8820: Refresh the process-group read allowlist. It has no commit of its own here: the allowlist is regenerated in this chain, and those regenerations already record the three reads that main added and drop the stale entry.
• Create the global memory buffer on first use #8026: Create the global memory buffer on first use. get_global_memory_buffer() in megatron/core/utils.py creates the buffer lazily, so a model built on its own collection no longer fails in its first sequence-parallel linear or local-attention forward.
• Create DDP's fallback expert data-parallel group collectively #8723: Create DDP's fallback expert data-parallel group collectively. When a collection lacks expt_dp, DDP setup now creates every single-rank group on all ranks with new_subgroups_by_enumeration, so group names match across ranks.
• Add test helpers to forbid global grid reads and build collections #8017: Add test helpers to forbid global grid reads and build collections. It adds the test-only helpers forbid_global_process_groups(), new_group_with_same_ranks() and build_test_pg_collection(), plus a grid_pg_collection fixture.
• Extract build_process_group_collection from initialize_model_parallel #8039: Extract build_process_group_collection from initialize_model_parallel. The builder returns (ProcessGroupCollection, StandardGridExtras) with the same groups, order and options as before, and sets no global.
• Add a dedicated fallback warning and process-group collection helpers #8049: Add a dedicated fallback warning and process-group collection helpers. It adds ProcessGroupFallbackWarning with per-call deprecated_in/removed_in versions, the require_process_groups and resolve_process_groups helpers, and checker support that counts resolve_process_groups calls as guarded reads.

Behavior changes for callers

• initialize_model_parallel() returns the ProcessGroupCollection instead of None. If building fails, no group is registered.
• initialize_model_parallel() no longer creates the global memory buffer. The private parallel_state._GLOBAL_MEMORY_BUFFER is removed, and _set_global_memory_buffer() no longer asserts that the buffer is new.
• On the DDP expt_dp fallback path, setup makes one new_group call per rank of the job instead of one.
• Fallback warnings use ProcessGroupFallbackWarning, a FutureWarning subclass, so existing FutureWarning filters still match. warnings.simplefilter("error", ProcessGroupFallbackWarning) makes every fallback raise.

The public names build_process_group_collection, StandardGridExtras, require_process_groups and resolve_process_groups need core approval before later PRs start calling them. At least 15 later PRs call resolve_process_groups.

Teams

core-adlr, core-nemo

Testing

Each member was GPU-tested against its base before consolidation. The consolidated chain (every unit in landing order, on current main) was run against main on 4 GPUs: 258 test files, and no test fails on the chain that passes on main. Tests in 28 files fail on both, mostly tests written for 8 GPUs, and one file times out on both.

🤖 Generated with Claude Code

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.