Skip to content

Core engine: HRX + HIP + NPU only (strip Vulkan, ROCm, ZINC, CUDA, ComfyUI; move site and kernel out) - #306

Merged
bong-water-water-bong merged 17 commits into
mainfrom
1bit/core-strip
Oct 5, 2026
Merged

bong-water-water-bong merged 17 commits into
mainfrom
1bit/core-strip

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Owner decision, 4 Oct: the engine is the HRX runtime, HIP kernels and the NPU. It is embedded in Lemonade, which brings its own llama.cpp, Vulkan, ROCm and CUDA backends. Do not merge yet: the release is on hold until the owner calls it.

This replaces the unmerged branches 1bit/hrx-only-stage3 and 1bit/npu-no-xclbin, rebased onto main 7eaa6e6 (one conflict, registry/architectures.json, resolved to stage 3's hrx-only sources with #302's pin). One commit per cut, so each can be reviewed alone.

Removed

  • Vulkan and ROCm/ROCmFPX (stage 3 + strip 1-2): both llama.cpp builds, --device vulkan|rocm, --lean, --long-model, DFlash2 on ROCm, H32, --prefill-device. Each removed device or flag is refused with a pointer to Lemonade's backends.
  • ZINC (strip 3): submodule, build, --device zinc, --zinc, registry column, packaging. Laya's GGUF candidates are now hrx + cpu.
  • CUDA (strip 4): DwarfStar's CUDA backend, the last CUDA build.
  • ComfyUI.cpp (strip 5): submodule, 1bit comfy.
  • NPU xclbin route: full ELFs only.
  • Site, blog, kernel build (strip 6): moved with history to 1bit-MONSTER/site (public, serving 1bit.gg since 4 Oct and building README + docs from an engine checkout) and 1bit-MONSTER/kernel. README, docs/ and context7.json stay here.

Kept

HRX (llama.cpp fork + hrx-system), the NPU (xdna-driver, full-ELF route), Laya, MoE NVMe streaming, registry + census, 1bit OS, Hyperloom, KernelForge, tokenizers, and:

  • DwarfStar, HIP only (strip 7): always its HIP backend for gfx1151, in build/ds4/hip.
  • ONNX (strip 8): Linux builds ryzenai-server from source. On Windows, 1bit.exe uses the ryzenai-server.exe Lemonade installs, under $LEMONADE_CACHE_DIR or %USERPROFILE%\.cache\lemonade, in bin\ryzenai-server\npu; build-windows.sh no longer builds one.
  • Windows build: the CPU llama-server plus --device onnx.

Verified

  • ryzen: default build, ctest 17/17 after every commit.
  • strixhalo: the weekly release configuration plus ONEBIT_LAYA_GGML (at e7ea91d) passes ctest 19/19:
    • --device hrx answers "Paris" on HRX0 (1 graph split, 203 tok/s), and auto routes to HRX.
    • The vulkan/rocm/zinc refusals and the comfy removal behave as intended.
    • The NPU GGUF route answers on full ELFs.
    • Laya end to end passes.
    • DwarfStar builds on HIP.
    • Max Tctl 71 °C.
  • Not yet verified: the Windows cross-build, and --device onnx against a real Lemonade install on Windows.

Follow-ups

  • Lemonade fork: 1bit-MONSTER/lemonade 1bit/onebit-core drops onebit's vulkan and cuda entries; pin it with this.
  • Delete the engine's old Pages deployment (its workflow is disabled and the 1bit.gg domain moved to the site repo).
  • NOTICE gaps that predate this PR: it lists mlir-aie and llvm-aie, which are not submodules, and is missing BusyBox and Dropbear, which ship in the 1bit OS image.

🤖 Generated with Claude Code

bong-water-water-bong and others added 17 commits October 4, 2026 15:47
The engine builds one llama.cpp: the HRX pin (third_party/llama.cpp), for
HRX0 and the CPU. Per the owner's 2026-10-01 direction (HRX + Loom or HIP
kernels, zero Vulkan, zero ROCm); Lemonade ships its own llamacpp backends
for Vulkan and ROCm.

Vulkan:
- third_party/llama.cpp-vulkan, cmake/vulkan.cmake, ONEBIT_VULKAN and
  .github/workflows/bump-llama-vulkan.yml are gone.
- The HRX build no longer turns on GGML_VULKAN (cmake/hrx.cmake); it also
  builds llama-quantize, which tools/hadamard_q4_0.py now uses.
- The Windows CPU llama-server is built from third_party/llama.cpp with
  GGML_HRX=OFF (scripts/build-windows.sh).
- registry/architectures.json loses its vulkan column: tools/registry_build.py
  reads only the HRX fork and ZINC; check_models.tsv and checked.json drop
  their vulkan rows; census.json is recomputed from its stored sweep.

ROCm:
- third_party/llama.cpp-rocmfpx, cmake/lean.cmake, ONEBIT_LEAN /
  ONEBIT_LEAN_ROCM and .github/workflows/bump-rocmfpx.yml are gone, with the
  long_route test (--long-model ran on the lean ROCm build).

Packaging, the weekly run, NOTICE and tools/weekly_changes.py follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`1bit serve` runs a .gguf on HRX0 or the CPU (and ZINC, DwarfStar, ONNX,
MLX, the NPU as before). RFC #213 stage 3.

Removed, each refused with the reason when asked for:
- --device vulkan and --device rocm (Lemonade's own llamacpp backend serves
  both);
- --prefill-device / --prefill-min-tokens (Vulkan decode with HRX prefill);
- --lean (ROCmFPX formats), --adaptive / --adaptive-at (Vulkan + ROCm
  overflow), --long-model / --long-from (the W4A4 long-prompt route);
- the DFlash2 one-server route: the ROCm recipes rotated-moe-ub1024 and
  rocm-dflash-p-min-0.4 (--dflash itself stays, on HRX and the CPU).

--device auto (what engine #271 kept on Vulkan):
- a plain .gguf: HRX0 where the build has HRX, else the CPU (CI, Windows);
- --mtp: HRX0 (it drafts there now, slower than Vulkan did);
- --mmproj: the CPU, with a note (HRX vision is an open RFC #213 gate;
  --device hrx asks for it);
- --parallel N > 1 on a gated delta-net model: refused (HRX runs one
  sequence; the CPU is no stand-in);
- an architecture our llama.cpp does not map (hrx_missing_arch: qwen4exp,
  zamba, zamba2, blackmamba and 13 upstream-only ones): refused on every
  device, pointing at Lemonade's llamacpp backend;
- a Hadamard-rotated Q4_0 (H32): auto's route (was the lean ROCm build).

--moe-slots is kept, code included: in a build without
ONEBIT_MOE_STREAMING_HRX it says it is not in this build and moves to HRX
with Qwen3.8-Flash-Next. The streamer lives only in the removed Vulkan pin.

The RAG servers (--embed / --rerank) run on HRX0, else the CPU. Laya offers
hrx (or cpu) and zinc. Tests follow: smoke_serve, parallel_args and
bench_selftest use --device cpu; device_route checks the new auto rules and
every refusal; hadamard_route, prism_route, recipes_route, laya_gguf_route
and laya_route_e2e drop their Vulkan and ROCm expectations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The docs, site, README and issue templates follow the build: no Vulkan or
ROCm llama.cpp, no --lean, --adaptive, --long-model or --prefill-device.
docs/vulkan.md and docs/lean.md are removed; the posts that link to them
point at their copies at 0baf286 (post text unchanged).

- serve.md: the device table, what --device auto does where HRX has no
  route, the removed devices and flags and what serve says for each,
  --moe-slots kept but not in this build.
- lemonade.md: Vulkan and ROCm are Lemonade's own llamacpp backends; the
  fork's onebit backend still lists vulkan (a follow-up there).
- registry.md: the vulkan column is gone; hrx also means the CPU route.
- hrx.md, moe-streaming.md, laya.md, recipes.md, releases.md, windows.md,
  onnx.md, PORTING.md: the same change where they named the removed builds.
- tools/site.py, context7.json, .pr_agent.toml, the issue templates and the
  issue agent's prompt: the device lists drop vulkan and rocm, add cpu.

Measurements taken on the removed builds are kept where they explain a
decision and are labelled as such.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…recipes)

Owner decisions of 2026-10-03 on the open questions of this branch.

- --mmproj under --device auto runs on HRX0. Unverified: vision on HRX is
  an open RFC #213 gate, to be checked on ZAYA1-VL; if it fails there, auto
  sends --mmproj back to the CPU. Comment and docs say so.
- --parallel N > 1 on a gated delta-net model (qwen35, qwen35moe,
  qwen3next) under auto: HRX0 with one slot and a warning on stderr (no
  multi-sequence delta-net on HRX yet), instead of a refusal. --device cpu
  still honors --parallel.
- H32 (Hadamard-rotated Q4_0, onebit.hadamard_q4_0) is dropped:
  tools/hadamard_q4_0.py, its serve route, the hadamard_q4_0 recipe match
  key and tests/hadamard_route.sh are removed, and serve refuses such a file
  by name. PrismML's own rotation (prism.hadamard, Ternary Bonsai) is a
  different route and keeps working (prism_route passes). The HRX build no
  longer needs llama-quantize.
- The refused architectures are generated, not listed by hand:
  tools/registry_build.py records every GGUF architecture the pinned fork's
  runtime builds (registry/architectures.json, gguf_architectures.hrx, 141
  at f5b7f4a), CMakeLists.txt compiles it into serve (hrx_archs.h), and
  serve refuses any other architecture on hrx and cpu.
- config/recipes.json has no built-in recipes: dflash-p-min-0 was a no-op
  on HRX (default p-min 0) and measured on the removed ROCm build.
  recipes_route now tests the mechanism with a recipe file.
- README: the census figures are the recomputed 94.71% mapped / 64.2%
  checked (334,413 models with an architecture).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fast-lane end-to-end test ran kernels derived from FastFlowLM and
compared against reference logits that are lost (owner, 2026-10-03).
tests/npu_lane_e2e.sh and its ctest registration (ONEBIT_NPU_REFERENCE)
are removed. The NPU check is now the private dx add-on's from-source
check, which is not in this repository.

docs/npu.md gets a "Checking the NPU" section; docs/kernel.md,
docs/PORTING.md and the bump-xdna / bump-linux PR text point at it instead
of the retired script. The kernel-free NPU tests (npu_full_elf_md5,
npu_q4nx, npu_pack_test, tokenizer_test) still run in CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The image carried the Vulkan llama-server and Mesa RADV; the engine has no
Vulkan build since RFC #213 stage 3. Owner decision 2026-10-03: rebuild it
around HRX.

os/mini/mkimage.sh:
- copies the HRX llama-server (build/hrx/llama/bin, at its build path, with
  every library it links) and the TheRock HSA runtime the build was
  configured with (ONEBIT_HRX_LIBHSA from CMakeCache.txt, at its path; 1bit
  has both compiled in), plus libdrm's amdgpu.ids where TheRock has one;
- drops RADV, the Vulkan loader and lavapipe;
- test images only: LLAMA_SERVER=<CPU llama-server> stands in for the HRX
  build and NO_XRT=1 leaves XRT out, for machines without TheRock or XRT;
- VMLINUZ= overrides the kernel image (where /boot is root-only).

os/mini/init.sh exports IREE_HAL_AMDGPU_LIBHSA_PATH and keeps HRX's Loom
kernel cache on the stick (GGML_HRX_JIT_CACHE_DIR=/data/cache/hrx-jit), so
only the first boot of an image compiles kernels; serves with --device auto
by default; the 1bit.check boot checks the HSA runtime. vmtest.sh serves
with --device cpu (the VM has no Radeon GPU). os/README.md lists what only
the hardware can check.

VM test on ryzen (CPU-only test image: LLAMA_SERVER from the HRX pin with
GGML_HRX=OFF, NO_XRT=1, kernel 7.0.0-34): UEFI boot from the USB image, SSH
in 9 s, data partition and HRX JIT cache dir mounted, 1bit serve
--device cpu answered "Paris." VMTEST PASS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NPU xclbin route was a comparison backend and nothing serves from it.
`1bit serve --device npu` (GGUF on the NPU) already called the forward with
full ELFs only, so this changes no served model or flag.

- I8Ctx (npu/forward/engine-src): drop init() (xrt::xclbin +
  register_xclbin + per-design instruction .txt), init_with_generator(),
  the xrt::kernel/instruction-BO launch branches, the HOST_ONLY group BOs
  and NPU_WBO_FLAGS. init_elf() is the only init.
- Q4nxNpuForward: drop the use_elf argument, the final_i8_*.xclbin /
  insts_i8_*.txt resolver and NPU_INFER_TAG.
- common.h: drop the FastFlowLM xclbin tables (XCLBIN_PATHS pointed at
  ~/amd-oss/fastflowlm), the full-ELF A/B name table and the xclbin
  kernel-argument/opcode constants. Nothing referenced them.
- device.h: removed. Never included and never implemented; its hwctx API
  configured a context from an xclbin path.
- model_config.h: drop the unused xclbin_* geometry fields.
- Comments, CMake help text and docs: say full ELF; the xclbin numbers in
  docs/npu.md stay as history and are labelled so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RFC #213 stage 3 already removed third_party/llama.cpp-vulkan, docs/vulkan.md, the bump
workflow and the Vulkan routes. What was left: tool examples naming Vulkan0, the Laya fetch
comment, and the long record of the removed --prefill-device split in docs/hrx.md (now a pointer
to its last version at 0baf286). serve still refuses --device vulkan/rocm and the removed flags
with the reason (Lemonade's llamacpp backend serves Vulkan and ROCm). The Windows package keeps
its CPU llama-server (GGML_VULKAN=OFF).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… at the ROCm build

RFC #213 stage 3 already removed third_party/llama.cpp-rocmfpx, ONEBIT_LEAN(_ROCM), docs/lean.md,
--lean/--device rocm/--long-model, the ROCm DFlash2 route, tools/hadamard_q4_0.py and the
ROCm recipes (config/recipes.json is empty). The one engine-side reference left was KernelForge's
README sending kept kernels back into ROCmFPX; they now go into an HIP kernel dispatched through
HRX. The ROCmFPX measurements stay as the record. TheRock (/opt/rocm-therock), which HRX itself
needs, stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner decision 2026-10-04: the engine is HRX + HIP kernels + the NPU; Lemonade's own backends
serve other GPUs. Removed: third_party/zinc, ONEBIT_ZINC(_BACKEND), scripts/build-zinc.sh,
docs/zinc.md, bump-zinc.yml, serve's --device zinc / --zinc route, the zinc registry column
(tools/registry_build.py, architectures.json, check_models.tsv, checked.json, census), the zinc
package and weekly build steps, the site nav entry, and NOTICE's ZINC attribution.

serve now refuses --device zinc and --zinc with the reason, as it does vulkan and rocm
(tests/device_route.sh checks both). With ZINC gone, Laya's .gguf candidates are hrx and cpu in
a build with HRX and cpu alone without; the Laya route tests use a code->hrx, rest->cpu policy.
architectures.json was edited in place (the zinc source, backends and counts dropped; the same
output tools/registry_build.py now writes); registry_pins and registry_gaps pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With ZINC gone, the engine's last CUDA build was DwarfStar's cuda backend. DwarfStar itself stays
(owner, 2026-10-04); only that backend leaves: ONEBIT_DS4_BACKEND and scripts/build-ds4.sh take
rocm (Strix Halo, the default), metal or cpu, and the Linux package no longer looks for a ds4 cuda
build. The ROCm backend is DwarfStar's own HIP code against TheRock's hipBLAS(Lt)/rocBLAS/rocWMMA/
hipCUB; it needs no engine-built ROCm llama.cpp.

Left on purpose: GGML_CUDA=OFF in cmake/hrx.cmake (it says the HRX build has no CUDA backend),
torch.cuda in the KernelForge drivers (PyTorch's device API, which ROCm torch implements on
HIP), and upstream ds4's own description. The Lemonade fork's onebit cuda backend entry lives
in that repository.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removed: third_party/comfyui.cpp (GPL-3.0, never linked), ONEBIT_COMFYUI, scripts/build-comfyui.sh,
docs/comfyui.md, the `1bit comfy` command (both the POSIX fexecve and the Windows spawn paths,
and the includes only they used), the weekly bump-comfyui check and fetch, the weekly-post and
site-nav entries, and NOTICE's ComfyUI.cpp attribution. The Windows package keeps serve (CPU,
ONNX) and route. The CMake banner that sat above ONEBIT_ONNX now describes ONNX.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The site and the kernel build now live in their own repositories (owner, 2026-10-04):
github.com/1bit-MONSTER/site (public, serving 1bit.gg; it builds README.md and docs/ from an engine
checkout, so those stay here) and github.com/1bit-MONSTER/kernel.

Removed: site/, blog/, tools/site.py, .github/workflows/pages.yml, third_party/linux (submodule
and .gitmodules entry), scripts/build-kernel.sh, config/kernel/, docs/kernel.md,
.github/workflows/bump-linux.yml, the weekly bump-linux check and linux fetch, the linux pin in
tools/weekly_changes.py, and blog/site entries in context7.json and .gitignore. README points to
both repositories; PORTING marks the kernel step as moved.

1bit OS keeps working unchanged: os/mini/mkimage.sh never used third_party/linux; it takes the
running kernel (or KVER=/VMLINUZ=). Only its comment and os/README.md now say the kernel packages
come from 1bit-MONSTER/kernel. NOTICE keeps the Linux kernel, now under 'Shipped in the 1bit OS
image only (built outside this repository)'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner, 4 Oct: DwarfStar is full HIP. ONEBIT_DS4 always builds DwarfStar's HIP backend
for gfx1151 (make strix-halo) into build/ds4/hip; the Metal and CPU options are gone
(CUDA went in core strip 4). build-ds4.sh takes only the prefix; weekly.sh's bump test
and package-linux.sh follow the new path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner, 4 Oct: use what Lemonade builds. Lemonade installs ryzenai-server.exe for its
ryzenai-server backend on Windows, so build-windows.sh no longer cross-builds one (nor
fetches Microsoft's ONNX Runtime GenAI / ONNX Runtime zips). 1bit.exe finds it under
$LEMONADE_CACHE_DIR, else %USERPROFILE%\.cache\lemonade, in bin\ryzenai-server\npu, then
on PATH; ONEBIT_ONNX_SERVER still overrides. Lemonade publishes that server for Windows
only, so Linux keeps building it from third_party/ryzenai-server (ONEBIT_ONNX).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DwarfStar's make ran with every core regardless of the caller's job cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@context7

context7 Bot commented Oct 4, 2026

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit eefde32

@bong-water-water-bong
bong-water-water-bong marked this pull request as ready for review October 5, 2026 11:20
@bong-water-water-bong
bong-water-water-bong merged commit 91dc6f9 into main Oct 5, 2026
7 of 10 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the 1bit/core-strip branch October 5, 2026 11:20
bong-water-water-bong pushed a commit that referenced this pull request Oct 5, 2026
The README still said the NPU layer kernel "is not yet built from source", while
docs/npu.md Step 3d on the same main documents the opposite: both the dense layer
kernel and lm-head run from source, ELF to ELF, with no xclbin at any point, one
compiled program covering every context position, verified on Qwen3-0.6B, 1.7B and
4B against an fp64 reference (cosine >= 0.9999, greedy decode exact).

Point the README at that, and keep the two qualifiers that are still true: the
kernel sources stay private per CONTRIBUTING, and the work is not yet merged here.

Recovered from the unreported docs-hrx-status branch, which is otherwise a
pre-RFC-#213-stage-3 snapshot (it would re-document the Vulkan, ROCm, ZINC, lean and
ComfyUI routes this repository removed, and it names docs/zinc.md, which #306 deleted).
This one commit was the only part still additive.
bong-water-water-bong added a commit that referenced this pull request Oct 5, 2026
The README still said the NPU layer kernel "is not yet built from source", while
docs/npu.md Step 3d on the same main documents the opposite: both the dense layer
kernel and lm-head run from source, ELF to ELF, with no xclbin at any point, one
compiled program covering every context position, verified on Qwen3-0.6B, 1.7B and
4B against an fp64 reference (cosine >= 0.9999, greedy decode exact).

Point the README at that, and keep the two qualifiers that are still true: the
kernel sources stay private per CONTRIBUTING, and the work is not yet merged here.

Recovered from the unreported docs-hrx-status branch, which is otherwise a
pre-RFC-#213-stage-3 snapshot (it would re-document the Vulkan, ROCm, ZINC, lean and
ComfyUI routes this repository removed, and it names docs/zinc.md, which #306 deleted).
This one commit was the only part still additive.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant