Skip to content

npu-lax: state the parity protocol (NPU routing adopted) - #81

Merged
bong-water-water-bong merged 1 commit into
mainfrom
lax-parity-protocol
Sep 25, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
lax-parity-protocol

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

The engine said "parity against fp64" for the 35B NPU decode without saying how the reference was run.

make_decode.py (OpenFlowLM-Next, at the engine's pin 2490fa6) re-uses the NPU's expert choice wherever a previous NPU run's router disagrees with the reference's own top-8. It prints each disagreement, and --strict-routing opts out. That's the right protocol for checking the arithmetic, because a near-tie in the 8th slot can't then look like a numerical error. But it doesn't measure the routing decisions themselves.

This PR says so in the README and docs/npu-lax.md. A --strict-routing run, which measures routing agreement, is a separate number and isn't recorded yet. No code change.

🤖 Generated with Claude Code

… adopts the NPU's routing)

make_decode.py re-uses the NPU's expert choice where the two routers disagree (--strict-routing
opts out), so the engine's corr / argmax numbers measure the arithmetic given the same experts.
The README and docs/npu-lax.md now say so; routing agreement is a separate measurement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@context7

context7 Bot commented Sep 25, 2026

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit 0143c7e

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Ambiguous Parity Claim

The documentation states that the NPU's arithmetic matches an fp64 reference "given the same routed experts" and that the reference adopts the NPU's expert choice where they differ. However, it does not clearly specify how the "same routed experts" are ensured or whether this is consistently applied across all test cases. This ambiguity could lead to inconsistent interpretation of what constitutes a valid parity check, especially when the NPU and reference disagree on expert selection.

It matches an fp64 reference given the same routed experts (the reference adopts the NPU's
expert choice where they differ; "Parity" below), and it decodes at 16.3-16.5 tok/s.
Missing Routing Agreement Measurement

The documentation notes that make_decode.py --strict-routing measures routing agreement, which is a separate metric from arithmetic parity. However, it also states that this routing agreement number is "not yet recorded here." This creates a disconnect between the documented protocol and the actual tracking of performance metrics, potentially leading to confusion about the completeness of the benchmarking effort.

decisions. `make_decode.py --strict-routing` keeps the reference's own top-8. A run with it
measures routing agreement, which is a separate number, not yet recorded here.

@bong-water-water-bong
bong-water-water-bong merged commit 8313ce0 into main Sep 25, 2026
5 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the lax-parity-protocol branch September 25, 2026 13:31
bong-water-water-bong added a commit that referenced this pull request Oct 5, 2026
…-Next) (#311)

* Bump HRX: llama.cpp e57beb9721af (fork #81-#83, #86-#88)

Moves third_party/llama.cpp (1bit-MONSTER/llama.cpp 1bit/hrx-vulkan-patched)
2e5fcf56a24d -> e57beb9721af; hrx-system stays at 51b1739ae5fd.
Registry regenerated at the new pin: 290 -> 292 architectures (qwen4exp).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Bump HRX: registry at the committed pin; qwen4exp is mapped now

registry/architectures.json was generated against the previous gitlink
(registry_pins failed). tests/device_route.sh used qwen4exp as its
"not built" example; at this pin Flash-Next runs on HRX, so the test now
checks it takes the plain route and keeps zamba2 as the refused example.
Docs that listed qwen4exp as unbuilt say it is back.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant