Repository navigation
docs: status after steps 1-3, HRX re-verified at the first daily bump - #16
Merged
Merged
Conversation
- README, PORTING, npu.md, lemonade.md: steps 1-3 landed (3a-3c; the layer kernel is still not built from source), the mlx recipe, the XDNA pin; Laya (step 4) is next. Drops stale "this PR" rows and "Not yet" items that steps 2 and 3 closed. - hrx.md: re-verified at llama.cpp f1a0aca / hrx-system 51b1739 (#13): e2e and ctest pass; fresh llama-bench numbers including BF16; new "Not yet": ggml-hrx accepts IQ3_XXS matmuls at scheduling time but cannot run them, so sub-4-bit UD files fail on HRX0 with no CPU fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong
force-pushed
the
docs/status-2026-09-23
branch
from
September 23, 2026 20:18
3a227c8 to
2d35ea8
Compare
bong-water-water-bong
pushed a commit
that referenced
this pull request
Oct 12, 2026
…l with a pre-fault count
Two findings from investigating a faulting run-7 arm, both bigger than the check I was doing.
1. THE FAULT IS TIMESTAMPED. server.log stamps the fault line in
minutes.seconds.milliseconds.microseconds: '0.07.279.709 E common_sampler_sample: HRX returned NaN
logits'. Format confirmed against that arm's own req=1 (7.28s is where a 1-request arm dies), not
assumed.
This removes a confound in MY OWN H2 test: a whole-window total cannot distinguish 'more evictions
caused the fault' from 'the fault caused more evictions', because a faulting arm crashes, may be
retried, and outlives its arm - so its window can simply be longer. Counting only up to the fault
instant is a causal test: a fault cannot cause evictions that precede it. 315-evict-per-arm.py now
prints pre-fault (causal) and window (continuity) columns, and refuses a causal conclusion when no
fault arm has a recoverable time.
2. TWO OF RUN 7'S THREE FAULTS WERE FACE A FOLLOWED BY A FULL SERVER CRASH. p7c and p7l logged the
NaN line then died - logs end in a gdb backtrace ('#16 _start', '[Inferior 1 detached]'). That is why
their detectors recorded transport_failures=1/RemoteDisconnected: the transport failure is DOWNSTREAM
of the NaN, not a separate failure mode. The console called both FaceA nan=1 and on the server side
that is right.
So run 7's faults are two Face-A NaN crashes (7.3s, 57.8s into their arms) and one Face-B parse
failure (81.2s, req=20).
VALIDATION: both faces matched, so a Face-B arm is not mistaken for one with no fault time. p3l
81.181s (Face B, via 'unparsed Content-only'), p7c 57.838s, p7l 7.280s. False-positive check across
ALL run-7 arms: exactly those 3 yield a time and 0 of the 8 clean arms do. A parser that found faults
in clean arms would fabricate the H2 effect, so that check is the one that matters.
bong-water-water-bong
pushed a commit
that referenced
this pull request
Oct 14, 2026
To run three probes back to back I wrote three scripts each shaped as 'while ps | grep <predecessor>; do sleep; done; clean; launch the next one'. Three disc sessions then appeared at once, including two copies of 315lout4, and 315chain-run.log was empty - stage 1 never started - while stage 2 launched anyway, because the guard is 'wait WHILE the predecessor is running', which is satisfied immediately when the predecessor does not exist at all. The guard answers 'is it running now?' rather than 'did it ever run and finish?'. Two sessions then composed the regime simultaneously, so neither arm's cell means what its label says and every arm in that window is unusable. Why this is the same defect one layer up: it is defect #14 with the sign flipped. There, absence of output was read as a clean result; here, absence of a process is read as completion. Both are a predicate on a proxy standing in for the event of interest, and both silently proceed. The campaign now has four instances of this shape - #13, where an EXTRA_ENV named a feature the build does not implement and absence of effect was read as absence of an effect; #14; #16; and the pkill-on-name traps of #11/#12 where killing the wrapper was read as stopping the run. Records the fix and what I did instead: the correct guard records evidence of the predecessor's completion via a sentinel line written after its last arm, and additionally requires that it started, but chaining is not worth that machinery here, so I abandoned it and ran the stages directly one at a time, verifying each launch by its own log header before starting the next - slower, but it cannot silently skip a stage, which is the failure mode that cost two composed regimes and about twenty minutes of box time this session is already over budget on. Cost: two simultaneous compositions whose arms are discarded, the section-40 chain probe never ran and must be re-issued, and one duplicate 315lout4 session, with no conclusions drawn from any arm in that window.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs catch-up after #9-#13 (and #15).
f1a0aca, hrx-system51b1739) on Strix Halo:tests/hrx_lemonade_e2e.shPASS (Paris on Vulkan0 and HRX0),ctest3/3. New llama-bench numbers (pp512/tg128): Q4_K_M Vulkan0 14097/349, HRX0 21990/314.unsupported HRX node ... iq3_xxs. So UD-Q2_K_XL, UD-IQ2_M and UD-IQ1_S fail on HRX0 with no CPU fallback. They run on Vulkan0.ZINC is not covered here: it waits for #14.
🤖 Generated with Claude Code