Repository navigation
Core engine: HRX + HIP + NPU only (strip Vulkan, ROCm, ZINC, CUDA, ComfyUI; move site and kernel out) - #306
Merged
Merged
Conversation
The engine builds one llama.cpp: the HRX pin (third_party/llama.cpp), for HRX0 and the CPU. Per the owner's 2026-10-01 direction (HRX + Loom or HIP kernels, zero Vulkan, zero ROCm); Lemonade ships its own llamacpp backends for Vulkan and ROCm. Vulkan: - third_party/llama.cpp-vulkan, cmake/vulkan.cmake, ONEBIT_VULKAN and .github/workflows/bump-llama-vulkan.yml are gone. - The HRX build no longer turns on GGML_VULKAN (cmake/hrx.cmake); it also builds llama-quantize, which tools/hadamard_q4_0.py now uses. - The Windows CPU llama-server is built from third_party/llama.cpp with GGML_HRX=OFF (scripts/build-windows.sh). - registry/architectures.json loses its vulkan column: tools/registry_build.py reads only the HRX fork and ZINC; check_models.tsv and checked.json drop their vulkan rows; census.json is recomputed from its stored sweep. ROCm: - third_party/llama.cpp-rocmfpx, cmake/lean.cmake, ONEBIT_LEAN / ONEBIT_LEAN_ROCM and .github/workflows/bump-rocmfpx.yml are gone, with the long_route test (--long-model ran on the lean ROCm build). Packaging, the weekly run, NOTICE and tools/weekly_changes.py follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`1bit serve` runs a .gguf on HRX0 or the CPU (and ZINC, DwarfStar, ONNX, MLX, the NPU as before). RFC #213 stage 3. Removed, each refused with the reason when asked for: - --device vulkan and --device rocm (Lemonade's own llamacpp backend serves both); - --prefill-device / --prefill-min-tokens (Vulkan decode with HRX prefill); - --lean (ROCmFPX formats), --adaptive / --adaptive-at (Vulkan + ROCm overflow), --long-model / --long-from (the W4A4 long-prompt route); - the DFlash2 one-server route: the ROCm recipes rotated-moe-ub1024 and rocm-dflash-p-min-0.4 (--dflash itself stays, on HRX and the CPU). --device auto (what engine #271 kept on Vulkan): - a plain .gguf: HRX0 where the build has HRX, else the CPU (CI, Windows); - --mtp: HRX0 (it drafts there now, slower than Vulkan did); - --mmproj: the CPU, with a note (HRX vision is an open RFC #213 gate; --device hrx asks for it); - --parallel N > 1 on a gated delta-net model: refused (HRX runs one sequence; the CPU is no stand-in); - an architecture our llama.cpp does not map (hrx_missing_arch: qwen4exp, zamba, zamba2, blackmamba and 13 upstream-only ones): refused on every device, pointing at Lemonade's llamacpp backend; - a Hadamard-rotated Q4_0 (H32): auto's route (was the lean ROCm build). --moe-slots is kept, code included: in a build without ONEBIT_MOE_STREAMING_HRX it says it is not in this build and moves to HRX with Qwen3.8-Flash-Next. The streamer lives only in the removed Vulkan pin. The RAG servers (--embed / --rerank) run on HRX0, else the CPU. Laya offers hrx (or cpu) and zinc. Tests follow: smoke_serve, parallel_args and bench_selftest use --device cpu; device_route checks the new auto rules and every refusal; hadamard_route, prism_route, recipes_route, laya_gguf_route and laya_route_e2e drop their Vulkan and ROCm expectations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The docs, site, README and issue templates follow the build: no Vulkan or ROCm llama.cpp, no --lean, --adaptive, --long-model or --prefill-device. docs/vulkan.md and docs/lean.md are removed; the posts that link to them point at their copies at 0baf286 (post text unchanged). - serve.md: the device table, what --device auto does where HRX has no route, the removed devices and flags and what serve says for each, --moe-slots kept but not in this build. - lemonade.md: Vulkan and ROCm are Lemonade's own llamacpp backends; the fork's onebit backend still lists vulkan (a follow-up there). - registry.md: the vulkan column is gone; hrx also means the CPU route. - hrx.md, moe-streaming.md, laya.md, recipes.md, releases.md, windows.md, onnx.md, PORTING.md: the same change where they named the removed builds. - tools/site.py, context7.json, .pr_agent.toml, the issue templates and the issue agent's prompt: the device lists drop vulkan and rocm, add cpu. Measurements taken on the removed builds are kept where they explain a decision and are labelled as such. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…recipes) Owner decisions of 2026-10-03 on the open questions of this branch. - --mmproj under --device auto runs on HRX0. Unverified: vision on HRX is an open RFC #213 gate, to be checked on ZAYA1-VL; if it fails there, auto sends --mmproj back to the CPU. Comment and docs say so. - --parallel N > 1 on a gated delta-net model (qwen35, qwen35moe, qwen3next) under auto: HRX0 with one slot and a warning on stderr (no multi-sequence delta-net on HRX yet), instead of a refusal. --device cpu still honors --parallel. - H32 (Hadamard-rotated Q4_0, onebit.hadamard_q4_0) is dropped: tools/hadamard_q4_0.py, its serve route, the hadamard_q4_0 recipe match key and tests/hadamard_route.sh are removed, and serve refuses such a file by name. PrismML's own rotation (prism.hadamard, Ternary Bonsai) is a different route and keeps working (prism_route passes). The HRX build no longer needs llama-quantize. - The refused architectures are generated, not listed by hand: tools/registry_build.py records every GGUF architecture the pinned fork's runtime builds (registry/architectures.json, gguf_architectures.hrx, 141 at f5b7f4a), CMakeLists.txt compiles it into serve (hrx_archs.h), and serve refuses any other architecture on hrx and cpu. - config/recipes.json has no built-in recipes: dflash-p-min-0 was a no-op on HRX (default p-min 0) and measured on the removed ROCm build. recipes_route now tests the mechanism with a recipe file. - README: the census figures are the recomputed 94.71% mapped / 64.2% checked (334,413 models with an architecture). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fast-lane end-to-end test ran kernels derived from FastFlowLM and compared against reference logits that are lost (owner, 2026-10-03). tests/npu_lane_e2e.sh and its ctest registration (ONEBIT_NPU_REFERENCE) are removed. The NPU check is now the private dx add-on's from-source check, which is not in this repository. docs/npu.md gets a "Checking the NPU" section; docs/kernel.md, docs/PORTING.md and the bump-xdna / bump-linux PR text point at it instead of the retired script. The kernel-free NPU tests (npu_full_elf_md5, npu_q4nx, npu_pack_test, tokenizer_test) still run in CI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The image carried the Vulkan llama-server and Mesa RADV; the engine has no Vulkan build since RFC #213 stage 3. Owner decision 2026-10-03: rebuild it around HRX. os/mini/mkimage.sh: - copies the HRX llama-server (build/hrx/llama/bin, at its build path, with every library it links) and the TheRock HSA runtime the build was configured with (ONEBIT_HRX_LIBHSA from CMakeCache.txt, at its path; 1bit has both compiled in), plus libdrm's amdgpu.ids where TheRock has one; - drops RADV, the Vulkan loader and lavapipe; - test images only: LLAMA_SERVER=<CPU llama-server> stands in for the HRX build and NO_XRT=1 leaves XRT out, for machines without TheRock or XRT; - VMLINUZ= overrides the kernel image (where /boot is root-only). os/mini/init.sh exports IREE_HAL_AMDGPU_LIBHSA_PATH and keeps HRX's Loom kernel cache on the stick (GGML_HRX_JIT_CACHE_DIR=/data/cache/hrx-jit), so only the first boot of an image compiles kernels; serves with --device auto by default; the 1bit.check boot checks the HSA runtime. vmtest.sh serves with --device cpu (the VM has no Radeon GPU). os/README.md lists what only the hardware can check. VM test on ryzen (CPU-only test image: LLAMA_SERVER from the HRX pin with GGML_HRX=OFF, NO_XRT=1, kernel 7.0.0-34): UEFI boot from the USB image, SSH in 9 s, data partition and HRX JIT cache dir mounted, 1bit serve --device cpu answered "Paris." VMTEST PASS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NPU xclbin route was a comparison backend and nothing serves from it. `1bit serve --device npu` (GGUF on the NPU) already called the forward with full ELFs only, so this changes no served model or flag. - I8Ctx (npu/forward/engine-src): drop init() (xrt::xclbin + register_xclbin + per-design instruction .txt), init_with_generator(), the xrt::kernel/instruction-BO launch branches, the HOST_ONLY group BOs and NPU_WBO_FLAGS. init_elf() is the only init. - Q4nxNpuForward: drop the use_elf argument, the final_i8_*.xclbin / insts_i8_*.txt resolver and NPU_INFER_TAG. - common.h: drop the FastFlowLM xclbin tables (XCLBIN_PATHS pointed at ~/amd-oss/fastflowlm), the full-ELF A/B name table and the xclbin kernel-argument/opcode constants. Nothing referenced them. - device.h: removed. Never included and never implemented; its hwctx API configured a context from an xclbin path. - model_config.h: drop the unused xclbin_* geometry fields. - Comments, CMake help text and docs: say full ELF; the xclbin numbers in docs/npu.md stay as history and are labelled so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RFC #213 stage 3 already removed third_party/llama.cpp-vulkan, docs/vulkan.md, the bump workflow and the Vulkan routes. What was left: tool examples naming Vulkan0, the Laya fetch comment, and the long record of the removed --prefill-device split in docs/hrx.md (now a pointer to its last version at 0baf286). serve still refuses --device vulkan/rocm and the removed flags with the reason (Lemonade's llamacpp backend serves Vulkan and ROCm). The Windows package keeps its CPU llama-server (GGML_VULKAN=OFF). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… at the ROCm build RFC #213 stage 3 already removed third_party/llama.cpp-rocmfpx, ONEBIT_LEAN(_ROCM), docs/lean.md, --lean/--device rocm/--long-model, the ROCm DFlash2 route, tools/hadamard_q4_0.py and the ROCm recipes (config/recipes.json is empty). The one engine-side reference left was KernelForge's README sending kept kernels back into ROCmFPX; they now go into an HIP kernel dispatched through HRX. The ROCmFPX measurements stay as the record. TheRock (/opt/rocm-therock), which HRX itself needs, stays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner decision 2026-10-04: the engine is HRX + HIP kernels + the NPU; Lemonade's own backends serve other GPUs. Removed: third_party/zinc, ONEBIT_ZINC(_BACKEND), scripts/build-zinc.sh, docs/zinc.md, bump-zinc.yml, serve's --device zinc / --zinc route, the zinc registry column (tools/registry_build.py, architectures.json, check_models.tsv, checked.json, census), the zinc package and weekly build steps, the site nav entry, and NOTICE's ZINC attribution. serve now refuses --device zinc and --zinc with the reason, as it does vulkan and rocm (tests/device_route.sh checks both). With ZINC gone, Laya's .gguf candidates are hrx and cpu in a build with HRX and cpu alone without; the Laya route tests use a code->hrx, rest->cpu policy. architectures.json was edited in place (the zinc source, backends and counts dropped; the same output tools/registry_build.py now writes); registry_pins and registry_gaps pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With ZINC gone, the engine's last CUDA build was DwarfStar's cuda backend. DwarfStar itself stays (owner, 2026-10-04); only that backend leaves: ONEBIT_DS4_BACKEND and scripts/build-ds4.sh take rocm (Strix Halo, the default), metal or cpu, and the Linux package no longer looks for a ds4 cuda build. The ROCm backend is DwarfStar's own HIP code against TheRock's hipBLAS(Lt)/rocBLAS/rocWMMA/ hipCUB; it needs no engine-built ROCm llama.cpp. Left on purpose: GGML_CUDA=OFF in cmake/hrx.cmake (it says the HRX build has no CUDA backend), torch.cuda in the KernelForge drivers (PyTorch's device API, which ROCm torch implements on HIP), and upstream ds4's own description. The Lemonade fork's onebit cuda backend entry lives in that repository. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removed: third_party/comfyui.cpp (GPL-3.0, never linked), ONEBIT_COMFYUI, scripts/build-comfyui.sh, docs/comfyui.md, the `1bit comfy` command (both the POSIX fexecve and the Windows spawn paths, and the includes only they used), the weekly bump-comfyui check and fetch, the weekly-post and site-nav entries, and NOTICE's ComfyUI.cpp attribution. The Windows package keeps serve (CPU, ONNX) and route. The CMake banner that sat above ONEBIT_ONNX now describes ONNX. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The site and the kernel build now live in their own repositories (owner, 2026-10-04): github.com/1bit-MONSTER/site (public, serving 1bit.gg; it builds README.md and docs/ from an engine checkout, so those stay here) and github.com/1bit-MONSTER/kernel. Removed: site/, blog/, tools/site.py, .github/workflows/pages.yml, third_party/linux (submodule and .gitmodules entry), scripts/build-kernel.sh, config/kernel/, docs/kernel.md, .github/workflows/bump-linux.yml, the weekly bump-linux check and linux fetch, the linux pin in tools/weekly_changes.py, and blog/site entries in context7.json and .gitignore. README points to both repositories; PORTING marks the kernel step as moved. 1bit OS keeps working unchanged: os/mini/mkimage.sh never used third_party/linux; it takes the running kernel (or KVER=/VMLINUZ=). Only its comment and os/README.md now say the kernel packages come from 1bit-MONSTER/kernel. NOTICE keeps the Linux kernel, now under 'Shipped in the 1bit OS image only (built outside this repository)'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner, 4 Oct: DwarfStar is full HIP. ONEBIT_DS4 always builds DwarfStar's HIP backend for gfx1151 (make strix-halo) into build/ds4/hip; the Metal and CPU options are gone (CUDA went in core strip 4). build-ds4.sh takes only the prefix; weekly.sh's bump test and package-linux.sh follow the new path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner, 4 Oct: use what Lemonade builds. Lemonade installs ryzenai-server.exe for its ryzenai-server backend on Windows, so build-windows.sh no longer cross-builds one (nor fetches Microsoft's ONNX Runtime GenAI / ONNX Runtime zips). 1bit.exe finds it under $LEMONADE_CACHE_DIR, else %USERPROFILE%\.cache\lemonade, in bin\ryzenai-server\npu, then on PATH; ONEBIT_ONNX_SERVER still overrides. Lemonade publishes that server for Windows only, so Linux keeps building it from third_party/ryzenai-server (ONEBIT_ONNX). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DwarfStar's make ran with every core regardless of the caller's job cap. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Docs7 for 1bit-monster/engine
Commit |
bong-water-water-bong
marked this pull request as ready for review
October 5, 2026 11:20
This was referenced Oct 5, 2026
Merged
bong-water-water-bong
pushed a commit
that referenced
this pull request
Oct 5, 2026
The README still said the NPU layer kernel "is not yet built from source", while docs/npu.md Step 3d on the same main documents the opposite: both the dense layer kernel and lm-head run from source, ELF to ELF, with no xclbin at any point, one compiled program covering every context position, verified on Qwen3-0.6B, 1.7B and 4B against an fp64 reference (cosine >= 0.9999, greedy decode exact). Point the README at that, and keep the two qualifiers that are still true: the kernel sources stay private per CONTRIBUTING, and the work is not yet merged here. Recovered from the unreported docs-hrx-status branch, which is otherwise a pre-RFC-#213-stage-3 snapshot (it would re-document the Vulkan, ROCm, ZINC, lean and ComfyUI routes this repository removed, and it names docs/zinc.md, which #306 deleted). This one commit was the only part still additive.
bong-water-water-bong
added a commit
that referenced
this pull request
Oct 5, 2026
The README still said the NPU layer kernel "is not yet built from source", while docs/npu.md Step 3d on the same main documents the opposite: both the dense layer kernel and lm-head run from source, ELF to ELF, with no xclbin at any point, one compiled program covering every context position, verified on Qwen3-0.6B, 1.7B and 4B against an fp64 reference (cosine >= 0.9999, greedy decode exact). Point the README at that, and keep the two qualifiers that are still true: the kernel sources stay private per CONTRIBUTING, and the work is not yet merged here. Recovered from the unreported docs-hrx-status branch, which is otherwise a pre-RFC-#213-stage-3 snapshot (it would re-document the Vulkan, ROCm, ZINC, lean and ComfyUI routes this repository removed, and it names docs/zinc.md, which #306 deleted). This one commit was the only part still additive. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Owner decision, 4 Oct: the engine is the HRX runtime, HIP kernels and the NPU. It is embedded in Lemonade, which brings its own llama.cpp, Vulkan, ROCm and CUDA backends. Do not merge yet: the release is on hold until the owner calls it.
This replaces the unmerged branches
1bit/hrx-only-stage3and1bit/npu-no-xclbin, rebased onto main 7eaa6e6 (one conflict,registry/architectures.json, resolved to stage 3's hrx-only sources with #302's pin). One commit per cut, so each can be reviewed alone.Removed
--device vulkan|rocm,--lean,--long-model, DFlash2 on ROCm, H32,--prefill-device. Each removed device or flag is refused with a pointer to Lemonade's backends.--device zinc,--zinc, registry column, packaging. Laya's GGUF candidates are now hrx + cpu.1bit comfy.Kept
HRX (llama.cpp fork + hrx-system), the NPU (xdna-driver, full-ELF route), Laya, MoE NVMe streaming, registry + census, 1bit OS, Hyperloom, KernelForge, tokenizers, and:
build/ds4/hip.1bit.exeuses theryzenai-server.exeLemonade installs, under$LEMONADE_CACHE_DIRor%USERPROFILE%\.cache\lemonade, inbin\ryzenai-server\npu; build-windows.sh no longer builds one.--device onnx.Verified
ONEBIT_LAYA_GGML(at e7ea91d) passes ctest 19/19:--device hrxanswers "Paris" on HRX0 (1 graph split, 203 tok/s), andautoroutes to HRX.comfyremoval behave as intended.--device onnxagainst a real Lemonade install on Windows.Follow-ups
1bit/onebit-coredrops onebit's vulkan and cuda entries; pin it with this.🤖 Generated with Claude Code