Repository navigation
cuda: share banked activation images - #148
Draft
GenerelSchwerz wants to merge 22 commits into
Draft
GenerelSchwerz wants to merge 22 commits into
GenerelSchwerz wants to merge 22 commits into
Conversation
Carry the reviewed kernel sources from PR125, PR129 and PR132 as the matching control for the separate tensor-affine HC follow-up. Assisted-by: Codex
Preserve gate intermediates, original rounding and live HC outputs. Retain strict input alias checks and standard allocation dependencies. Assisted-by: Codex
Use the exact tested control sources from published PR130, PR129 and PR132. Assisted-by: Codex
Reuse the allocated affine match to find the planned HC_POST and retain prepared emission priority. Reject expanded normalization cuts whose terminal is not written by this kernel. Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
This was referenced Oct 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Share the original activation image when multiple ordinary
MUL_MATnodes read the same contiguous F32 tensor with bank or sample dimensions. Use the existing reuse planner, image-family keys, overwrite guards and stream handling. Keep the original quantization, matrix kernels and NVFP4 row-scale layout.This follow-up is tested against #147 at
7ef1d05483aad1e3bf974c2cfe6b1f0ba7f41496. The branch includes earlier series dependencies; the leaf changes one host file (24 additions, 11 deletions) and adds eight calls to the existing matrix test (6 lines).Existing prepared norm writers keep their singleton image contract. Banked norm output can share the ordinary converter image. Strided banked inputs, input writes, incompatible image families and disabled fusion retain their original behavior. This leaf does not extend routed
MUL_MAT_ID, remove a multi-consumerREPEAT, or integrate a private executor.On RTX5070Ti/SM120a, six alternating pairs of 2,000 captured replays per build gave these median complete-component wall times in microseconds. Each of the four material reductions was faster in all six pairs; the two flat cases are not claimed as gains.
Validation used the original CPU limits (5e-4; native FP4 2e-2), four updated datasets and byte-identical candidate/control GPU outputs, CPU references and uploaded inputs. The six utility pairs had 104 stage comparisons; 20 boundary pairs had 488. Coverage includes eight readers (144 converter calls reduced to 18), separate bank/sample axes, MMQ layouts, tails, native/forced activation precision, normal GPU/CPU scheduler allocation and rebuilds, live inputs, weight broadcasting and negative stride/write/family/disable cases. The original prepared flat path preserved 16 fused emissions and two separate conversions: two updated datasets intentionally use nonzero SCALE bias, which the existing emitter rejects.
Each build passed 268 existing regression cases and eight matrix-pair whole-graph tests, with the same six original unsupported TQ skips. Ten serial sanitizer runs were clean and checked 136 CPU stages. Fresh host compiles passed for SM75/80/89/90 (compile-only). All 26 measured component/boundary CUDA arenas had zero delta. The fresh host object's existing shader remained identical in resources and encoded instructions; 143 CUDA objects were reused literally, no device kernel was added and CUDA library size growth was zero.
All builds and GPU checks held the ordered shared build/GPU locks for each complete batch. An initial qualification stopped on the incorrect flat-emitter count; its 77 successful commands were independently revalidated, followed by 28 fresh continuation commands. No source, arithmetic, ownership check or tolerance changed during that correction.
These are isolated component results, not model-serving, MTP, peak-VRAM, additive series gains or older-GPU runtime results.
AI disclosure: Codex implemented the scoped change and prepared the validation and description under the fork owner's explicit authorization; a bounded GPT-6.1 Sol review independently checked source and raw evidence.