Server: 1bit-server speaks Lemonade's backend protocol

[bong-water-water-bong]

1bit-server lets Lemonade run the engine through the same backend protocol it uses for its GGUF recipes (llamacpp, llamacpp-hrx). Only the protocol is shared. The model code, tokenizer, sampler and template rendering are all this repo's.

What

• 1bit-server:
  • Flags: takes Lemonade's launch flags (-m --ctx-size --device --port --jinja --metrics --parallel 1).
  • /health: 503 Loading model while loading, then 200.
  • Endpoints: /v1/models, /v1/chat/completions and /v1/completions, streaming (SSE, [DONE]) and not.
  • Telemetry: OpenAI usage plus llama-server timings, the fields Lemonade's telemetry reads.
• Chat:
  • the GGUF's own Jinja template, with chat_template_kwargs (e.g. enable_thinking)
  • <think> routed to reasoning_content, which Lemonade's chat, MCP and Ollama paths read
  • stop strings, and end-of-turn detection
• Prompt cache: the shared prefix with the previous request is reused (cache_n), so multi-turn chat evaluates only the new turn.
• Pinned deps: three header-only MIT libraries in cmake/deps.cmake, each with a sha256: nlohmann/json 3.12.0, cpp-httplib 0.57.1 and minja 021c2293. They're credited in NOTICE.
• Docs: docs/server.md covers the contract, the verification, and what's not done yet.

Verification

• server_e2e.py: launches the binary with Lemonade's flags and checks over HTTP (Qwen3-0.6B, strixhalo):
  • greedy /v1/completions text byte-identical to HF transformers' fp32 greedy continuation
  • streamed == non-streamed, for completions and chat (content and reasoning)
  • prefix reuse (cache_n = n_prompt - 1)
  • stop strings, enable_thinking:false, and the thinking split
  • the 400/404 error codes
• golden_template_qwen3: 12 conversations byte-identical to HF apply_chat_template, including tools, tool calls, tool responses and the thinking toggle. Runs in CI.
• Unit tests: test_text_stream covers UTF-8, stops and reasoning byte by byte; test_openai checks response shapes structurally; plus test_sampler. All run in CI.
• Totals: ctest 10/10 on strixhalo, 8/8 locally; zero warnings.

Not yet (in docs/server.md)

Tool-call output parsing, /v1/responses, embeddings, /metrics, >1 concurrent sequence, and non-CPU devices. The NPU and GPU backends plug in behind this server next.

🤖 Generated with Claude Code

[bong-water-water-bong]

Closing: wrong direction. The repo ports the existing 1bit-MONSTER engine instead (embedded Lemonade, HRX with Vulkan, the NPU engine, Laya); see the port plan.