Merge upstream llama.cpp (2026-10-08) and the final halo-box/llama.cpp master (re-open of #155)

[dzannotti]

Re-opens #155, which was reverted in #168 because agents stopped working on master after it merged. The content is unchanged; this PR is where the breakage gets found and fixed before it lands again.

---

halo-box/llama.cpp is archived, so this fork now syncs with ggml-org/llama.cpp directly. This PR does that in two real merges, per SYNC.md:

1. halo-box/llama.cpp's final master, which includes its 2026-10-07 upstream sync (halo server : do not re-verify replayed draft tokens after a checkpoint restore #34) and those resolutions.
2. The 39 upstream commits after that.

Afterwards strix is 0 commits behind upstream. Merge with "Create a merge commit", not squash, or the upstream history is lost and the next sync conflicts on every commit again.

Draft until two things are done (see the end of this description): the remaining backend-ops runs pass, and someone runs Qwen3.8-Flash on HIP and Vulkan.

Resolution rules

MTP follows upstream's design. Everywhere else, strix's side wins where the two collide, upstream's non-conflicting changes stay, and upstream fixes are ported into strix's code. Strix contributors: please check how your areas came out.

qwen4exp / QSA

• strix's QSA stack is kept whole in qwen4exp.cpp and its models.h struct: selected-key/WMMA block selection, incremental pooled keys, PLE prefetch and the HIP prefill work.
• llama-memory-hybrid-idx carries both APIs: strix's qsa_* for qwen4exp, and upstream's k-pool for the new GLM5Next model (GLM-5.3-Flash). strix's context set_input_qsa, which upstream had deleted, is restored.

MTP

• Kept: strix's graph_mtp. It builds the same draft as upstream's Qwen4Exp: add MTP ggml-org/llama.cpp#29761: eh_proj of [enorm(e); hnorm(h)], the draft's own nextn.hc_head_* mixer, sparse QSA attention. It also loads shared sidecars that ship without embeddings or an LM head.
• From upstream: the converter, gguf-py tensor maps and arch tables. New conversions follow upstream's layout, and strix loads both layouts.

Recurrent state (llama-graph)

Upstream's single fixed-size state gather (ggml-org#29856) is combined with strix's in-place gated delta net. In-place-capable consumers keep the full gather, and their state view sits next to the consumer whether or not the rows are in place. In-place and gathered ubatches then build the same graph. Without that, every switch between them reallocated the graph: a speed cost in normal use, and an abort under GGML_SCHED_DEBUG_REALLOC in both upstream's shared-seq test and strix's seq-cp graph-reuse test.

CUDA / HIP

• Kept as strix's: top-k (HIP radix), argsort, gated delta net, and the RDNA3.5 fused paths.
• MMQ: upstream's out-of-bounds fix and host-side tile choice (CUDA: fix MMQ out-of-bounds reads ggml-org/llama.cpp#29953), with strix's RDNA3.5 tile overrides on top. strix's fused paths pad src1 for the largest tile, replacing the get_J_max helper upstream removed.
• Added: upstream's shared-expert MoE fusion, next to strix's fused GDN decode.
• Updated to upstream's new signatures: strix's call sites for the norm, mmvf and chunk-row helpers. Upstream's CCCL version-guard fix is applied to strix's argsort.

Vulkan

• Kept as strix's: flash_attn.comp, flash_attn_cm2.comp, lightning_indexer.comp and topk_nary_search.comp, with upstream's FA shared-memory out-of-bounds fix (vulkan: fix Flash Attention shmem write out of bounds ggml-org/llama.cpp#29988) ported in.
• Left out: upstream's sparse FA (vulkan: sparse flash attention for quantized K/V ggml-org/llama.cpp#29639, vulkan: extend sparse FA support to coopmat2 ggml-org/llama.cpp#30003), because strix had deleted its shaders. Upstream's lightning-indexer rewrite (qwen4exp : halve the indexer score memory ggml-org/llama.cpp#29825) is reverted on the Vulkan host side to match strix's shader; its CUDA and Metal parts stay.

Allocation

• Upstream's alloc_buffer_n refactor replaces strix's backport of the view-init fix. Its planner flushed on zero-size tensors, which left views of a tensor larger than max_size uninitialised. That is fixed here, caught by strix's test_view_init_after_max_size_split, and is worth upstreaming.

Server and tests

• Server: upstream's final server : preserve context checkpoints across slot save/restore ggml-org/llama.cpp#26004 (checkpoints across slot save/restore) replaces strix's earlier port of it, keeping strix's on-device restore flags. Speculative accept keeps strix's replay branch, then upstream's rejection sampling at temp > 0.
• Tests: both sides' new tests are kept, and strix's rollback test is migrated to common_batch / llama_process.

Known regression to port later

None open. ggml-org#29856 was reverted in the first merge commit and restored in the second, so review the two together.

Verified

• Builds: Vulkan (RADV, Radeon 8060S) and HIP (gfx1151, ROCm 10.0), no new warnings.
• ctest: 68/69 on both backends. The one failure is test-jinja-py, because the image has no jinja2.
• test-recurrent-state-rollback-dsv4: passes on both backends here, but fails intermittently, as it does on pristine strix (3 of 4 runs).
• test-backend-ops against CPU on ROCm0: TOP_K, ARGSORT, RMS_NORM, GATED_DELTA_NET, PAD and ROLL have no failures.

Before taking this out of draft

• The remaining test-backend-ops runs pass: MUL_MAT, MUL_MAT_ID and FLASH_ATTN_EXT on ROCm0; TOP_K, FLASH_ATTN_EXT, LIGHTNING_INDEXER and GATED_DELTA_NET on Vulkan0.
• Qwen3.8-Flash on HIP and Vulkan, with and without MTP (--spec-type draft-mtp, including a shared-sidecar MTP file), compared against current master for prompt-processing and token-generation speed and output. Nothing has been benchmarked yet.

🤖 Generated with Claude Code

[gaetan-puleo]

Please don't merge this 🙏 , I am working on a way to create better merge request for merging upstream