[jonathan308]

Update 2026-10-21: merged main

The head is now ad596687: main merged in (mlx#36, thread-scoped command-buffer commit limits, the Nemotron 3.5 Super native prerequisite). No conflicts. scripts/check_forkdiff.py passes (190 changed files, all described). The Metal library built from this tree is unchanged (156f2624…); #36 changes host code only.

Update: production footprint charges restored

The head is now 6ee3062c. 4a355746 reverts 99dc5f42 (the one-page minimum in get_allocation_footprint_policy()), so the Metal footprint policy is again {page, page, 0, 0, page}, the same as the core the current provider release ships. 6ee3062c keeps docs/src/usage/unified_memory.rst in fork.yaml, because the capped no-copy paragraph still differs from upstream.

Why: with the page minimum, the SDK's allocation estimates for MTP drafters that declare no buffer layout (Qwen4 inline MTP, NemotronH35 MTP) grow from about 4.5 KB to 37–671 MB per token, so those requests can never be admitted, and Qwen3.5 MTP's partitioned buffers cost about 32 KB more per token. This release is for GLM-5.3-Flash and must not change how other models are charged. Nothing in the provider or the SDK installs the allocation cap whose page-backed fallback the minimum covered.

Effect on GLM: the serving MTP charge is 4.05 GB per request (4.89 GB with the minimum). It was rechecked on one M5 Ultra with the rebuilt chain (d-inference ml-explore#1516 15d7a833d, darkbloom 94a83e27…, metallib 156f2624… unchanged): admission flood, concurrency, 125K context, SSD prefix restore across a restart and a 20-minute soak. See ml-explore#1516.

The source otherwise matches 2bba342d: git diff 2bba342d 6ee3062c touches only mlx/backend/metal/allocator.cpp (one line), its docs paragraph, the standalone cap test cases that asserted the minimum, and fork.yaml.

---

Summary

The single-Mac native core for GLM-5.3-Flash. It holds the ten general commits from darkbloom/sealed-all-sum (#35) that one-Mac serving needs, replayed onto main, with none of the two-Mac code.

Until now, the single-Mac release (d-inference #1516, mlx-swift-lm #314) pinned core 3a179119. That commit exists only on the cluster stack (#33 → #35), so the single-Mac release could not merge before the JACCL and sealed all-sum work. This branch removes that dependency.

Commit	Cherry-picked from	Change
b7434f7f	ccf517a0	Revert the NAX gather_qmm simdgroup skip (ml-explore#4352) on this tree
0077571e	eb576eb2	Bound sorted quantized NAX tail loads
6beeabd8	65eb14fa	Test contiguous sorted quantized activation tiles
e3799f23	f7bbf7d7	Give floating-point NAX loaders a distinct Metal symbol
d02ac777	1fb86d3f	Separate incompatible quantized Metal loader implementations
9266fda0	adbc349b	Fix legacy quantized partial-tile bounds
2a0c9bbb	9e22e199	Optional Metal backing caps; FP32 CPU quantized sums
99dc5f42	4d913204	Cover page-backed small buffers in detached Metal footprints
23164b8b	54975cd0 (Metal half only)	Prefault capped no-copy backing; validate custom launch dimensions
2bba342d	3a179119	Retry 512-thread kernels with explicit compiler capacity

54975cd0 mixed two things: the JACCL startup-wait budget and Metal fixes. Only its Metal half is here. The startup-wait budget and its group API stay on darkbloom/sealed-all-sum, together with the JACCL send-frame/progress-guard/GID fixes, the upstream JACCL backports and the sealed all-sum.

Before

flowchart LR
  R["Single-Mac GLM release<br/>#1516 / #314"] --> S["mlx-swift bfaa8a90"]
  S --> C["core 3a179119<br/>(cluster branch only)"]
  C --> J["JACCL fixes + sealed all-sum<br/>#33 → #35"]
  J --> M["main"]
  classDef block fill:#fee2e2,stroke:#b91c1c,color:#450a0a
  class C,J block

Loading

After

flowchart LR
  R["Single-Mac GLM release<br/>#1516 / #314"] --> S["mlx-swift work/glm53-flash-release"]
  S --> C["core work/glm53-flash-release<br/>(this PR)"]
  C --> M["main"]
  T["Cluster stack #33 → #35"] -.->|"same general patches, rebased later"| M
  classDef safe fill:#dbeafe,stroke:#2563eb,color:#172554
  class R,S,C safe

Loading

Evidence

• Same tree as the cluster pin, minus the cluster code. git diff 3a179119 2bba342d -- . ':!mlx/distributed' ':!tests/jaccl' ':!fork.yaml' lists only six cluster-only files: the distributed and environment-variable docs, mlx/backend/cpu/distributed.cpp, and three distributed Python test files including the scoped_env helper. mlx/distributed/ and tests/jaccl/ are identical to main. Every Metal kernel source matches 3a179119 byte for byte.
• Fork-diff gate: scripts/check_forkdiff.py --upstream-ref upstream/main passes, with 189 changed files and all 189 described. fork.yaml is the 3a179119 version with three sections removed: JACCL, two-rank sealed all-sum, and the upstream JACCL fixes.
• Metallib: scripts/fetch-metallib.sh in the d-inference [BUG] vmap over function returning constant value has incorrect output shape ml-explore/mlx#1516 worktree builds mlx.metallib from this branch with sha256 156f2624f9f1b1413dfabd8b6e16423121b0e5b18e2573c23c4e023b606a462d. That is byte-identical to the metallib built from 3a179119 on the same Mac and toolchain (Xcode 27.1).
• Downstream build: d-inference [BUG] vmap over function returning constant value has incorrect output shape ml-explore/mlx#1516 (provider, -c release, Xcode 27.1 / Swift 6.4) builds and links on the full single-Mac chain (mlx 2bba342d, mlx-c 2f1bcab9, mlx-swift cf921c05, SDK 3ab6dd68). darkbloom sha256 is e863e668…. No model was loaded with this build; end-to-end single-Mac qualification is the next gate.

Relationship to the cluster stack

#35 keeps its commits unchanged. When this PR merges, the cluster stack has to merge main. Its general commits carry the same patches as the ones here, so the code conflicts should be trivial; fork.yaml will conflict and needs resolving by hand. The cluster pair (tp/pair.sh) must be re-verified bitwise after that merge.

Merge danger

Door: two-way. This is a cherry-pick of commits already reviewed on #35, with no new source.

Blast radius: quantized Metal kernels (tail bounds and loader symbols), the optional allocation cap (off unless installed), footprint quotes and the custom-kernel launch retry. Kernel sources are the ones ml-explore#1516 already pinned through 3a179119.

🤖 Generated with Claude Code