site: the 1bit.MONSTER template, a cleaned-up blog, three new posts

.github/workflows/pages.yml

@@ -22,9 +22,9 @@ name: pages
 on:
   push:
     branches: [main]
-    paths: [README.md, "docs/**", "site/**", tools/site.py, .github/workflows/pages.yml]
+    paths: [README.md, "docs/**", "blog/**", "site/**", tools/site.py, .github/workflows/pages.yml]
   pull_request:
-    paths: [README.md, "docs/**", "site/**", tools/site.py, .github/workflows/pages.yml]
+    paths: [README.md, "docs/**", "blog/**", "site/**", tools/site.py, .github/workflows/pages.yml]
   workflow_dispatch:
 
 permissions:

blog/2026-09-24-first-week-milestones.md

@@ -0,0 +1,52 @@
+<!--
+Copyright 2026 bong-water-water-bong
+SPDX-License-Identifier: Apache-2.0
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+tags: milestones, npu, hrx, vulkan, zinc
+
+# The first week of 1bit engine
+
+The clean repository started on 2026-09-22. In its first three days, the engine came up on every device it targets, inside Lemonade, with each step verified on real hardware before it merged. These are the milestones, with the numbers and where they come from.
+
+## The NPU, on full ELFs generated in C++
+
+The XDNA 2 NPU now runs without an xclbin and without per-context kernel files. The engine generates each kernel's full ELF (the design plus its control code) in C++ when the model loads, and XRT opens it directly; for each token, the host submits every layer and the lm head as one runlist ([#8](https://github.com/1bit-MONSTER/engine/pull/8), [#9](https://github.com/1bit-MONSTER/engine/pull/9)).
+
+- Qwen3-0.6B decodes at **91 tok/s** (11.0 ms per token), and **87 tok/s** served through Lemonade.
+- The XDNA driver and its XRT are pinned to upstream and built privately ([#12](https://github.com/1bit-MONSTER/engine/pull/12)).
+- The layer kernel itself is being rebuilt from scratch as our own open kernel ("dx", on IRON and Peano). RMSNorm, RoPE and attention pass their tests so far, correctly rounded to bf16.
+
+## HRX and Vulkan in one build, then both at once
+
+The Radeon iGPU has two devices in one llama.cpp build: `HRX0` on AMD's HRX runtime and `Vulkan0`. HRX follows AMD's live ggml-hrx, pinned to the pair AMD tests and bumped daily ([#11](https://github.com/1bit-MONSTER/engine/pull/11)); Vulkan has its own pin on upstream llama.cpp's latest release ([#28](https://github.com/1bit-MONSTER/engine/pull/28)).
+
+- **HRX went from 3-5x slower than Vulkan to close behind it.** Its prefill is now 1.3-1.7x Vulkan's, and its decode within 10% on 4-bit files.
+- **Our patches make HRX correct on more models** ([#29](https://github.com/1bit-MONSTER/engine/pull/29)): an IQ3_XXS kernel, and op claims that only take the nodes HRX can actually run. `test-backend-ops` on HRX0 went from about 1150 failures to **791 passing, 0 failing**, and Unsloth's sub-4-bit files, which failed on HRX, now come within 1% of Vulkan's perplexity.
+- **Prefill on HRX, decode on Vulkan, with zero copies** ([#30](https://github.com/1bit-MONSTER/engine/pull/30)): a whole 8192-token request on Qwen2.5-7B finishes **26% sooner**. [How it works](2026-09-24-hrx-prefill-vulkan-decode.md).
+
+## Beyond AMD
+
+- **NVIDIA, through ZINC's CUDA backend** ([#14](https://github.com/1bit-MONSTER/engine/pull/14)): Qwen3.5-9B at **167-173 tok/s** on an RTX 5090, and 295 tok/s through ZINC's Vulkan backend on Strix Halo.
+- **Apple Silicon**, through Lemonade's `mlx` recipe and lemon-mlx-engine, verified on an M4 ([#10](https://github.com/1bit-MONSTER/engine/pull/10)).
+
+## Underneath
+
+- **Tokenizers:** Hugging Face `tokenizers` v0.23.2 behind our C ABI reads any model's `tokenizer.json`, byte-exact on 18 models ([#19](https://github.com/1bit-MONSTER/engine/pull/19)).
+- **Quantization:** on the GPU, Unsloth's Dynamic UD-Q4_K_XL beats Q4_K_M: 17% lower KL divergence for 2% more size. On the NPU, a ternary model converts to 4-bit exactly, and XDNA 2 multiplies int8 by int4 natively ([wiki](https://github.com/1bit-MONSTER/engine/wiki/Quantization)).
+- **The Linux kernel** is pinned to upstream v7.3-rc4 with `amdxdna` in-tree ([#17](https://github.com/1bit-MONSTER/engine/pull/17)), and the **Laya router**'s source and checkpoints are pinned and hash-verified ([#18](https://github.com/1bit-MONSTER/engine/pull/18)).
+
+## Next
+
+Step 4 is the Laya router, which decides where each request runs. After it: every Hugging Face architecture, kept current by a daily census. The [porting map](../docs/PORTING.md) tracks it, and the [wiki](https://github.com/1bit-MONSTER/engine/wiki) keeps every measured number with how it was taken.

blog/2026-09-24-hrx-prefill-vulkan-decode.md

@@ -0,0 +1,55 @@
+<!--
+Copyright 2026 bong-water-water-bong
+SPDX-License-Identifier: Apache-2.0
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+tags: hrx, vulkan, zero-copy
+
+# Prefill on HRX, decode on Vulkan, one KV cache
+
+On Strix Halo's Radeon iGPU, AMD's HRX runtime prefills a prompt faster than Vulkan, and Vulkan decodes faster than HRX. `1bit serve --device vulkan --prefill-device hrx` now uses both on the same request, and the key-value cache never gets copied between them.
+
+## The idea
+
+A request has two phases. **Prefill** reads the whole prompt at once and is compute-bound; **decode** makes one token at a time and is bound by memory bandwidth. Our one llama.cpp build carries both GPU backends, `HRX0` and `Vulkan0`, so the engine loads the model on each, runs the prompt prefix on HRX, and hands the rest of the request to Vulkan.
+
+The hand-off is the hard part. Prefill's output is the KV cache: every layer's keys and values for every prompt token. Copying it from one backend to the other would cost the time the split saves.
+
+## Zero copies
+
+So both contexts use **one** KV cache:
+
+- HRX allocates it in its own device memory and exports it as a **dma-buf** (through HSA).
+- Vulkan imports the same memory with `VK_EXT_external_memory_dma_buf` and reads it in place, at 110.6 GB/s against 113.2 GB/s for its own allocations.
+- The only thing that moves between them is the cache's bookkeeping: which cells hold which positions of which sequence.
+
+We tried sharing host memory first. It worked, but it cost Vulkan half its decode speed; device memory through a dma-buf does not.
+
+Both sides compute the cache's layout independently, so they must agree on it exactly: tensors on 4 KiB boundaries, split into chunks of at most 1 GiB (Vulkan reads garbage from a single buffer past about 4 GiB), and a hash of the layout checked on both sides. HRX's prefill is fastest in whole batches, so only whole ubatches of the prompt go to HRX, and short prompts skip the split: below 1024 tokens (`--prefill-min-tokens`) it does not pay.
+
+## Measured
+
+llama-server on Strix Halo, prefill and whole request:
+
+| Model | Prompt | Vulkan alone | HRX prefill + Vulkan |
+|---|---|---|---|
+| Qwen2.5-7B Q4_K_M | 2048 | 1541 / 4314 ms | **1078** / 3890 ms (-10%) |
+| Qwen2.5-7B Q4_K_M | 8192 | 7347 / 10376 ms | **4646** / 7668 ms (**-26%**) |
+| Qwen3-0.6B Q4_K_M | 8192 | 1289 / 2267 ms | **1089** / 2032 ms (-10%) |
+
+Decoding on the shared cache runs within 3% of Vulkan with its own cache. The output is the same model: against Vulkan alone, teacher-forced over 64 tokens, the mean KL divergence is 0.0002-0.0006 nats and the top token agrees on 64-65 of 65 positions. For scale, Q4_K_M quantization itself is 0.063 against BF16.
+
+## Where it lives
+
+The patches are in our llama.cpp fork, on top of AMD's tested HRX commit, and are rebased onto every new AMD pair automatically: dma-buf export in ggml-hrx, dma-buf import in ggml-vulkan, the cache sharing in llama, and the prefill hand-off in llama-server. The weights are loaded twice, once per device, so it suits machines with memory to spare; Strix Halo's unified memory is one. The details are in the [HRX docs](../docs/hrx.md).

blog/2026-09-24-new-direction-from-1bit-monster.md

@@ -0,0 +1,46 @@
+<!--
+Copyright 2026 bong-water-water-bong
+SPDX-License-Identifier: Apache-2.0
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+tags: announcement, engine
+
+# 1bit.MONSTER is now 1bit engine
+
+1bit.MONSTER grew fast: an inference engine, a model zoo, benchmarks, a store, a voice assistant and a lot of experiments, all in one repository with all of its history. The engine that came out of it is now its own project, **1bit engine**, and this site is its home.
+
+## What changed
+
+**A clean repository.** The engine lives in [1bit-MONSTER/engine](https://github.com/1bit-MONSTER/engine). It holds verified code only, without the development history. Each piece is ported over once it passes its end-to-end test on Strix Halo, and the [porting map](../docs/PORTING.md) shows what has moved and what is next.
+
+**Apache-2.0.** The old repository was GPL-3.0. The engine is Apache-2.0, with a copyright and license notice on every file, enforced in CI, and every third-party project it builds on credited in its NOTICE.
+
+**Inside Lemonade, not around it.** The old engine vendored its own copy of Lemonade. Now [Lemonade](../docs/lemonade.md) is the server you talk to (its catalog, downloads, router and UI), and it runs the engine as one of its backends, through `1bit serve`, the same way it runs `llama-server`.
+
+**Upstream, pinned and kept current.** The XDNA driver and XRT, AMD's HRX pair, llama.cpp for Vulkan, ZINC, the Linux kernel, Hugging Face tokenizers and the Laya router are each pinned to upstream, and the pins are bumped automatically, so new models and fixes arrive soon after upstream ships them.
+
+## What it runs on
+
+One `1bit serve` drives:
+
+- the [XDNA 2 NPU](../docs/npu.md), on full ELFs generated in C++;
+- [HRX and Vulkan](../docs/hrx.md) on the Radeon iGPU, one llama.cpp build with both;
+- [ZINC](../docs/zinc.md), which also reaches NVIDIA GPUs through CUDA;
+- [MLX](../docs/apple.md) on Apple Silicon.
+
+Measured numbers live on the [wiki](https://github.com/1bit-MONSTER/engine/wiki). [The first week of 1bit engine](2026-09-24-first-week-milestones.md) lists what the first three days produced.
+
+## The old site
+
+The 1bit.MONSTER site and its posts stay online in the [1bit.MONSTER archive](https://1bit-monster.github.io/1bit-MONSTER/), and old links land on a page that points at their archived copy. The blog page here lists the old posts worth keeping. 1bit JARVIS, the voice assistant, is not part of the engine for now; it will come back later as its own piece.

blog/archive.tsv

@@ -0,0 +1,15 @@
+2026-09-02	Byte-identical: 1,000 tokens of proof against AMD's real runtime	1bit-post-byte-identical.html
+2026-09-02	The 35B MoE's NPU packing: 5,120-byte tiles, 4,736-byte rows, and an A/B interleave	1bit-post-35b-moe-layout.html
+2026-09-01	Zaya 8B Q4NX on the AMD NPU	1bit-post-zaya-q4nx-hrx.html
+2026-08-29	HRX: we put AMD's experimental IREE runtime inside our engine	1bit-post-hrx-engine.html
+2026-08-25	One engine, every model	1bit-post-1775-models.html
+2026-08-25	What 1-bit actually does to a model	1bit-post-1bit-explained.html
+2026-08-25	ROCm vs CUDA, from the 1-bit trenches	1bit-post-rocm-vs-cuda.html
+2026-08-25	Local RAG with our own embeddings	1bit-post-local-rag.html
+2026-07-30	Qwen3.6-35B-A3B streams fully on the Ryzen AI NPU	1bit-post-qwen36.html
+2026-07-26	Zyphra family complete	1bit-post-zyphra.html
+2026-07-24	One binary, all formats	1bit-post-binary-formats.html
+2026-07-23	Unsloth for AMD	1bit-post-unsloth.html
+2026-07-11	TokenRouter: token-level multi-backend inference	1bit-post-tokenrouter.html
+2026-07-06	Three bugs that broke 97 tok/s	1bit-post-three-bugs.html
+2026-07-02	NPU v12: M=32 batch decode at 97 tok/s	1bit-post-npu-v12.html