Repository navigation
Bump HRX: llama.cpp f1a0aca141de, hrx-system 51b1739ae5fd - #13
Merged
Merged
Conversation
bong-water-water-bong
deleted the
bump-hrx/f1a0aca141de-51b1739ae5fd
branch
September 23, 2026 19:21
bong-water-water-bong
pushed a commit
that referenced
this pull request
Sep 23, 2026
- README, PORTING, npu.md, lemonade.md: steps 1-3 landed (3a-3c; the layer kernel is still not built from source), the mlx recipe, the XDNA pin; Laya (step 4) is next. Drops stale "this PR" rows and "Not yet" items that steps 2 and 3 closed. - hrx.md: re-verified at llama.cpp f1a0aca / hrx-system 51b1739 (#13): e2e and ctest pass; fresh llama-bench numbers including BF16; new "Not yet": ggml-hrx accepts IQ3_XXS matmuls at scheduling time but cannot run them, so sub-4-bit UD files fail on HRX0 with no CPU fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong
added a commit
that referenced
this pull request
Sep 23, 2026
…#16) - README, PORTING, npu.md, lemonade.md: steps 1-3 landed (3a-3c; the layer kernel is still not built from source), the mlx recipe, the XDNA pin; Laya (step 4) is next. Drops stale "this PR" rows and "Not yet" items that steps 2 and 3 closed. - hrx.md: re-verified at llama.cpp f1a0aca / hrx-system 51b1739 (#13): e2e and ctest pass; fresh llama-bench numbers including BF16; new "Not yet": ggml-hrx accepts IQ3_XXS matmuls at scheduling time but cannot run them, so sub-4-bit UD files fail on HRX0 with no CPU fallback. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.MONSTER> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 7, 2026
bong-water-water-bong
pushed a commit
that referenced
this pull request
Oct 13, 2026
…perturbation level that yields Face A Two attempts produced only Face B, and the second is informative: attempt 1 got no fingerprint lines at all (defect #13, the rig's default build does not implement the feature), attempt 2 with the correct build gave 6/6 arms Face B at req=1-11, and attempt 3 - the same 64-name list plus COPY_PROBE/SENTINEL/LOGITS_FP - reached req=59/66 with 58 OK and ONE Face B fault. That last one matters: the extra environment moved the fault from every arm at req=1-11 back toward the intermittency the objective describes, still Face B. The remaining difference from the configuration that produced Face A at scale is the selector's SIZE: 315nan read tail:16 - sixteen tensors per graph - and recorded 16 Face-A faults in 17 arms, while these attempts read 64 named tensors per graph. So the working hypothesis is a dose-response in instrumentation, with more host-side reads per graph shifting the fault from Face A toward Face B. Third attempt: same environment, build and readout, with the selector reduced to about 16 l_out layers sampled across the model (l_out-0, l_out-3, ..., l_out-45, plus 46 and 47). This keeps the property the sweep depends on - l_out-<layer> is the last node of its layer's chunk, hence faithfully readable - while matching the perturbation level that produced Face A. Readout unchanged, with the added check that Face B arms again mean the sweep is mis-targeted rather than every layer clean. A 16-layer sample brackets the origin layer rather than pinpointing it, which is the intended resolution here. Declares this the last selector attempt: if it also yields Face B, the honest conclusion is that the l_out selector and the Face-A fault are not co-observable at any perturbation level tried, recorded as a finding about the instrument rather than as a negative result about layers.
bong-water-water-bong
pushed a commit
that referenced
this pull request
Oct 13, 2026
Chasing defect #15 turned up a second operability problem. The frozen rig gates every arm on load1 < 4 (315-disc.sh:91), and against the current tenancy that threshold is unreachable: loadavg sits at 4.72 with tctl at 44.5C while our own side-processes are verified at zero (evictonly=0, CPU workers=0) and GTT is at 26GiB, i.e. the box is as quiet as we can make it and the gate still refuses. The load is other agents' workloads - clickhouse, redis, dockerd, node. Two consequences. First, a run can stall indefinitely without failing: a swap script of mine sat in exactly this loop with every one of our preconditions satisfied, and only a self-imposed iteration cap let it proceed, while the rig's own gate has no such cap - and an arm that never starts is indistinguishable from outside from an arm that ran and produced nothing, the same ambiguity this campaign has hit twice before in defect #13's inert EXTRA_ENV and #14's NANONLY sweep that could not distinguish clean from not-run. Second, the gate is a property of the original tenancy: load1 < 4 and tctl < 80 were calibrated for a box this campaign had to itself, so with other agents running servers and CPU work the gate measures THEIR load and silently changes what in-regime means for our arms - not the GTT regime itself, since arms carry their own gtt_start, but the ambient condition the campaign has treated as constant, which is the baseline against which section 39's inducing effect and the whole ambient-vs-forced comparison are drawn. Not resolved here: the honest statement is that this campaign's ambient baseline and its arm-starts are now conditional on a box it no longer exclusively owns, and no arm in this record was tagged with the foreign load present when it ran. The rig's newer 'resident_inventory (foreign, not ours)' line is a step toward recording that and should be read as required context for any arm from this session onward.
bong-water-water-bong
pushed a commit
that referenced
this pull request
Oct 14, 2026
To run three probes back to back I wrote three scripts each shaped as 'while ps | grep <predecessor>; do sleep; done; clean; launch the next one'. Three disc sessions then appeared at once, including two copies of 315lout4, and 315chain-run.log was empty - stage 1 never started - while stage 2 launched anyway, because the guard is 'wait WHILE the predecessor is running', which is satisfied immediately when the predecessor does not exist at all. The guard answers 'is it running now?' rather than 'did it ever run and finish?'. Two sessions then composed the regime simultaneously, so neither arm's cell means what its label says and every arm in that window is unusable. Why this is the same defect one layer up: it is defect #14 with the sign flipped. There, absence of output was read as a clean result; here, absence of a process is read as completion. Both are a predicate on a proxy standing in for the event of interest, and both silently proceed. The campaign now has four instances of this shape - #13, where an EXTRA_ENV named a feature the build does not implement and absence of effect was read as absence of an effect; #14; #16; and the pkill-on-name traps of #11/#12 where killing the wrapper was read as stopping the run. Records the fix and what I did instead: the correct guard records evidence of the predecessor's completion via a sentinel line written after its last arm, and additionally requires that it started, but chaining is not worth that machinery here, so I abandoned it and ran the stages directly one at a time, verifying each launch by its own log header before starting the next - slower, but it cannot silently skip a stage, which is the failure mode that cost two composed regimes and about twenty minutes of box time this session is already over budget on. Cost: two simultaneous compositions whose arms are discarded, the section-40 chain probe never ran and must be re-issued, and one duplicate 315lout4 session, with no conclusions drawn from any arm in that window.
bong-water-water-bong
pushed a commit
that referenced
this pull request
Oct 14, 2026
…NT and the run was aimed at the wrong one Caught while preparing the run; it would have made the campaign UNINTERPRETABLE rather than wrong, which is the worse failure because it looks like a null. THE DEFECT: instrumentation is split across two source trees that do not overlap. llama-315copy carries the FINGERPRINT (hrx-tensor-fp, hrx-finite-fp; built by 315fp-build, 315fp2-build, 315fbar-build); llama-315probe carries the PRODUCER PROBE (hrx-producer, GGML_HRX_PRODUCER_PROBE/DEVICE; built by 315probe-build). Checked in the artefacts, not inferred: 315probe-build's libggml-hrx.so.0.18.0 has 7 hrx-producer strings and 0 hrx-tensor-fp strings, and llama-315copy's graph-executor.cpp has 0 occurrences of kProbeSentinel|producer_probe_target|hrx-producer. WHAT IT WOULD HAVE DONE: the campaign set GGML_HRX_TENSOR_FP=<15-name list> on a 315probe-build binary where that variable is INERT - so the 15-name list, which is the fault-yield enabler, would not have been active. Both conditions would have degenerated to 'probe, with and without a coherence bit' at an unknown (probably ambient ~5%) fault rate with 4 eviction arms each - landing in the pre-registration's own INVALID/under-powered row, while a reader saw two arms with no faults and no fingerprint lines. THE AMENDMENT IS A TARGET CHANGE, NOT A REDESIGN: A'/B' needs three things from its binary - the fingerprint (yield enabler + onset readout), the coherence toggle (the intervention) and the writeback instrument (the warmth check). Two of three live in llama-315copy, so the fix is to patch THAT tree - the toggle's anchor is unique there because the only other params block is DEVICE_LOCAL - and build 315fp-build. Verified after patching: fingerprint, GGML_HRX_STAGING_COHERENT, and GGML_HRX_WRITEBACK_FINGERPRINT (2 occurrences in command-program-executor.cpp). THE PRODUCER PROBE IS DROPPED FROM THIS CAMPAIGN, deliberately: not needed for A'/B', not in this tree, and leaving GGML_HRX_PRODUCER_PROBE=1 in the environment of a binary without it would have been a second inert variable - so those two variables are REMOVED rather than left in and ignored. The device-probe question keeps its own pre-registration and its own tree. THE GENERALISABLE TRAP, recorded because it will recur: an environment variable whose instrument is not compiled into the binary is SILENTLY INERT and nothing in the run's output says so. The only defence is to check the ARTEFACT (strings <lib> | grep <env name>) rather than the source tree you believe you built. Same class as the record's defect #13 ('the rig's default BIN build does not implement GGML_HRX_TENSOR_FP at all, so an EXTRA_ENV-driven sweep was silently inert') - the same trap a second time, which is why it is written down rather than just fixed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves both submodules to the pair ROCm/ggml-staging-automation now pins.
f1a0aca141def1a0aca141defab1624aa92551b1739ae5fdCI here builds without HRX. Before merging, on Strix Halo:
cmake -B build -G Ninja -DONEBIT_HRX=ON && cmake --build build --target onebit && tests/hrx_lemonade_e2e.sh build/1bit build/hrx/llama/bin/llama-server