Repository navigation
ggml-hrx: HIP kernels through HRX (plumbing + add-on hook) - #86
Merged
bong-water-water-bong merged 4 commits intoOct 5, 2026
Merged
Conversation
Each ggml/src/ggml-hrx/hip/kernels/*.hip is compiled by amdclang++ into a raw code object per GGML_HRX_HIP_TARGETS entry (default gfx1151) and embedded in the library. HRX loads it with hrx_executable_load_data, the same call the Loom JIT result goes through. Neither the HIP runtime nor ggml-hip is used. - hip-kernel-registry: a HipKernel (bindings, uint32 launch parameters, workload parameters, launch-geometry function) becomes a KernelDefinition in family "hip". resolve_kernel_definition finds it there. - hip-kernel-loader: KernelExecutableCache::get_or_compile takes the embedded code object and the launch geometry in place of a Loom JIT compile. - hip-dispatches.cpp: the one place HIP matchers are registered. - Example: hip_scale_f32 + dispatch-hip-scale.cpp (GGML_OP_SCALE), opt-in with GGML_HRX_HIP_EXAMPLE_SCALE=1. - ggml-hrx-hip-smoke: loads, dispatches and checks a code object using libhrx only. - hip/README.md: the recipe. Upstream files get one-line hooks only (CMakeLists include, the resolve fallback, the get_or_compile branch, a friend declaration and the dispatch registration call). Checks on strixhalo (gfx1151), performance power mode (120 W): - ggml-hrx-hip-smoke, 6 runs: axpy 0/1000003 mismatches, LDS + wave32 shuffle block sum max relative error 0 over 3907 blocks. Pointer args, by-value args and hidden kernargs (blockDim/gridDim) all work. - test-backend-ops -o SCALE -b HRX0 with the HIP matcher, 3 runs: 4/4 passed each run. GGML_HRX_LOG_DISPATCH shows hip.scale_f32 matched. - ZAYA1-8B Q4_K_M llama-bench -fa 1 -p 512 -n 128 -r 3, two interleaved rounds against the same tree without this change: pp512 2037.7/2050.4 before vs 2052.3/2053.3 after; tg128 93.0/93.0 vs 92.9/92.7. No regression with HIP matchers off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kernels and matchers can live outside the tree. With -DGGML_HRX_HIP_ADDON_DIR=<dir>: - <dir>/kernels/*.hip are compiled and embedded with hip/kernels/*.hip (same targets and flags, looked up by stem; configure fails on a repeated stem); - <dir>/*.cpp and *.h are compiled into ggml-hrx with GGML_HRX_HIP_ADDON defined, and register_hip_dispatches calls the add-on's ggml_hrx_hip_addon_register after the matchers in hip/; - <dir>/addon.cmake is included if present (extra tools). Without the option the build is unchanged: only the example matcher. The attention-sink rescale that follows FlashAttention gets a generic hook (set_hip_attention_sink_hook): an add-on kernel can take the rescale, otherwise the Loom kernel runs as before. README: the add-on layout, and the registry trap that every Fused registration of a root op is tried before any SingleOp one regardless of priority. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Output sizes 128 and 384, K 256/1024/2560, for HRX HIP kernels that take output % 128 == 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lities) supports_op only hands a node to the dispatcher when eager_capability_declared lists its op, so a HIP matcher for an op outside that list never fired: the node went to another backend. hip_declare_eager_op(op), called from a matcher's register_* function while the registry is built, adds the op to a set that eager_capability_declared consults for every op it does not know (one line in its default branch). hip_eager_op_declared builds the registries on first use, so the set is filled before the scheduler's first probe. Without a declaring matcher the set is empty and the build behaves as before; the public tree names no add-on op. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Author
|
Added 3527b56
|
bong-water-water-bong
marked this pull request as ready for review
October 5, 2026 11:20
bong-water-water-bong
merged commit Oct 5, 2026
001832c
into
1bit/hrx-vulkan-patched
10 of 26 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lets HRX run hipcc code objects as ggml-hrx kernels, beside the Loom kernels, and lets an external kernel add-on plug in at build time. Draft: do not merge yet. The engine release is on hold.
616034bHIP plumbing:hip/ggml-hrx-hip.cmakecompileship/kernels/*.hipwith amdclang++ into raw code objects perGGML_HRX_HIP_TARGETS(default gfx1151) and embeds them in the library.hip-kernel-registry(aHipKerneldescription becomes a KernelDefinition in familyhip), andhip-kernel-loader(the executable cache loads the embedded object instead of a Loom compile).hip-dispatches.cpp.README.mdwith the recipe; examplehip_scale_f32with its matcher (opt-inGGML_HRX_HIP_EXAMPLE_SCALE=1); theggml-hrx-hip-smoketool.8719cf4add-on hook:GGML_HRX_HIP_ADDON_DIR=<dir>adds<dir>/kernels/*.hipto the code-object build and compiles<dir>/*.cpp/*.hinto ggml-hrx withGGML_HRX_HIP_ADDONdefined.<dir>/addon.cmakeif present and callsggml_hrx_hip_addon_register(registry).set_hip_attention_sink_hook. The dispatch falls back to Loom when no hook is set.d5de4b1: threeMUL_MAT_VEC_FUSIONcases (one-token Q4_K/Q5_K gate/up + SwiGLU at outputs 128/384).Checks on strixhalo (gfx1151), performance mode:
ggml-hrx-hip-smokePASS;test-backend-ops -o SCALE -b HRX04/4 on 3 runs withhip.scale_f32matched, and 4/4 with the example off.🤖 Generated with Claude Code