Repository navigation
STRATA_GGML_DIR with llama.cpp b11580 fails CUDA MMQ precision/template interfaces #2251
Description
Strata v0.1.43.1 (a0116b2782a5b82622358c04a06496485d12b2d6) builds with its shipped llama.cpp dependency (3cf03257f219afbe7334045ff7c6a06ac68c627d). This report concerns overriding that dependency through STRATA_GGML_DIR with llama.cpp b11580 (4413a192410fad1f054e9216114087a3e01527a0). The override's ggml CPU library compiles successfully, but Strata's CUDA MMQ integration does not compile against the newer internal interfaces.
Minimal reproduction on Linux with CMake, a C++ compiler, and CUDA 13.3 installed:
git clone https://github.com/Niko1221/Strata.git
git -C Strata checkout a0116b2782a5b82622358c04a06496485d12b2d6
git clone https://github.com/ggml-org/llama.cpp.git
git -C llama.cpp checkout 4413a192410fad1f054e9216114087a3e01527a0
cmake -S Strata -B build -DCMAKE_BUILD_TYPE=Release \
-DSTRATA_ENABLE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 \
-DSTRATA_GGML_DIR="$PWD/llama.cpp" -DSTRATA_BUILD_TESTS=OFF \
-DSTRATA_MMQ_KQUANTS=OFF -DSTRATA_PREFILL_MMQ=OFF
cmake --build build --target strata -j 4Sanitized decisive diagnostics (ten compiler errors total):
src/prefill/moe_mmq.cu(153): error: no instance of overloaded function "ggml_cuda_mmq_get_config" matches the argument list
src/prefill/moe_mmq.cu(207): error: no instance of function template "mul_mat_q_case" matches the argument list
The second error repeats for the other quantization cases at lines 208–215. The current bridge calls ggml_cuda_mmq_get_config(type, J, fallback, cc) and mul_mat_q_case<type>(ctx, args, stream). At b11580 these interfaces require a ggml_prec argument and a second template parameter, respectively. The new header explicitly instantiates the ordinary variants with GGML_PREC_Q8. Strata already quantizes its activation input through quantize_mmq_q8_1_cuda; selecting Q8 preserves that representation rather than enabling the new FP4 activation path. The newer header also exposes ggml_cuda_mmq_needs_fallback(nrows_x) for the tile eligibility check.
Sources: Strata bridge, new MMQ header.
Could Strata support this newer dependency through the existing override, or document the supported override revisions? This is not a failure of the shipped pin or the CPU library itself. An independently built CUDA vision helper using b11580 also builds successfully. We are preparing a narrow maintained bridge update with numerical and task checks while upstream compatibility catches up; no kernel rewrite or llama.cpp source change is proposed.
Activity
Tsury commentedon Oct 13, 2026
A source check also found that the MMQ argument layout changed: its final field is now
J_best, rather than the precedingncols_opt. A precision-only bridge adjustment compiles but the existing synthetic IQ3_S numerical test aborts withJ_best=230. Moving the preceding upstream tile-selection rule into the bridge caller, and passingGGML_PREC_Q8throughout, resolves that failure. With current b11582 (af50f2f248b475d9760f8e67503d01c0f194f5f4, MMQ header identical to b11580), both tested IQ3_S/IQ4_NL and IQ3_S/Q2_0 MMQ output matrices are byte-identical to the shipped dependency reference: 2560 routed rows by 2560 values per matrix, all finite. The Q2_0 double-reference and affected CPU parity checks also pass. Matched native task checks are still running. llama.cpp sources remain unmodified. Relevant caller migration: ggml-org/llama.cpp#29953 .Tsury commentedon Oct 13, 2026
The complete maintained Strata 0.1.43.1 native/CPU build with stock b11582 is now in use. The bridge preserves Q8 activation precision and the preceding tile-selection rule while supplying the new precision/template and J_best arguments. Both selected synthetic IQ3_S MMQ matrices remain byte-identical to the shipped-pin reference, with every value finite and per-row checks. Three balanced pairs for each of two native coding families, plus warm tool/edit continuations, passed 12/12 first natural phases per arm with identical initial request payloads. Matched fresh/reused prompt controls show similar timings; we claim no useful speed gain or broad reliability result. The existing frontend and independently qualified b11580 vision helper stay fixed. llama.cpp sources remain unmodified. We will remove the local bridge delta when upstream supports the newer dependency with equivalent checks.
Tsury commentedon Oct 13, 2026
The same caller compatibility work also applies to official Strata 0.1.44 (
ed4773eb7c8f1f1531ba2bc70e0e1987de2661af) with stock llama.cpp b11601 (ec0bfac1794ad722221dd68d84760f8980355fb2). Tagged source still uses the preceding interfaces. The newly added NVFP4 and K-quant translation units add callers beyond the originalmoe_mmq.cubridge.The maintained rebase preserves the preceding tile-selection rule and supplies
J_best. Q8 branches use explicitGGML_PREC_Q8in configuration selection andmul_mat_q_caseinstantiation. The existing optional W4A4 branch usesGGML_PREC_Q4and the newDECL_MMQ_CASE_W4A4macro. Affected caller files aresrc/prefill/moe_mmq.cu,mmq_kq_lib.cu,mmq_nvfp4_w4a8.cu, andmmq_nvfp4_w4a4.cu. No llama.cpp source modification was needed.The complete native/CPU engine and CUDA vision helper build with CUDA 13.3.73, SM120, and the stock b11601 sources. The selected IQ3_S/IQ4_NL and IQ3_S/Q2_0 synthetic MMQ matrices each contain 6,553,600 finite floats and remain byte-identical to the retained maintained b11582 reference. Affected Q8/IQ CPU parity and the existing Q2_0 reference check pass. Three fresh vision-helper starts and repeated encodes also match the retained b11580 output bytes for the same 36-token image fixture. These checks do not validate NVFP4 or K-quant numerical behavior, general quality, or a speed gain; native task checks remain separate.
I am adding the CUDA evidence here rather than presenting a cross-backend PR for review. The contribution instructions require builds for every touched backend, and this assignment provides CUDA hardware only; HIP/SYCL builds remain unverified. The small caller rebase is still maintained locally while the upstream dependency pin catches up.
Niko1221 commentedon Oct 19, 2026
Thanks for taking the time to report this.
We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to read properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.
To reopen it takes about two minutes:
/reopen. A check runs and either reopens the issue or lists what is still missing.The rules are in CONTRIBUTING.md.