1bit serve: forward llama-server's other routes (Lemonade checklist #3)

[bong-water-water-bong]

Fixes item 3 of the Lemonade embedding checklist (#68).

The gap: Lemonade's llamacpp backend, which the onebit recipe inherits, forwards /v1/responses, /tokenize and /slots to the backend, and 1bit serve answered 404.

The fix: on vulkan, hrx and rocm, 1bit serve now forwards these to the llama-server behind the model:

• /v1/responses, streamed or not;
• /tokenize, /detokenize, /apply-template;
• GET /slots, POST /slots/<id>?action=… (query string kept), GET /props, GET /metrics.

On npu, zinc and mlx these routes answer 501 with a clear message. docs/serve.md lists them.

Measured on Strix Halo, Lemonade's test/server_llm.py --wrapped-server onebit on the pinned fork:

device	before	after
Vulkan	19 / 31	23 / 31
HRX	19 / 31	23 / 31

The four newly passing tests: Responses API, Responses API streaming, slots, tokenize. Direct checks: /tokenize returns token ids, /slots lists the slots, /props returns 200. A slot action returns llama-server's own 501 ("start it with --slot-save-path").

🤖 Generated with Claude Code

[context7]

Docs7 for 1bit-monster/engine

Result	Status	Action
Deployment	➖ Not used	—
Content review	➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan.	—

Commit 631c08c

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 631c08c)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 68 - Partially compliant Compliant requirements: Fix the gap for /v1/responses, /tokenize, /detokenize, /apply-template, /slots, /props, /metrics routes Implement forwarding logic for vulkan, hrx, rocm devices Implement 501 responses for npu, zinc, mlx devices Update docs/serve.md with the new routes and their behavior PR description includes measured results showing improvement from 19/31 to 23/31 tests passing Non-compliant requirements: None Requires further human verification: Verification that the measured performance improvements (19/31 to 23/31) are accurate on Strix Halo hardware Confirmation that the new routes work correctly with the actual Lemonade integration

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Potential Query String Encoding Issue The query string encoding logic in the relay function may not properly handle all special characters in query parameters. Specifically, the use of httplib::encode_query_component might not cover all edge cases that could occur in real-world query strings, potentially leading to malformed requests being forwarded to the backend. if (!req.params.empty()) { std::string query; for (const auto& [k, v] : req.params) query += (query.empty() ? "?" : "&") + httplib::encode_query_component(k) + "=" + httplib::encode_query_component(v); path += query; } Possible Race Condition in Model Loading The first and first_json handlers check ready status before forwarding requests, but there's a potential race condition where the model could become ready between the check and the actual relay. This could lead to inconsistent behavior when the model is loading. if (!ready) { r.status = 503; r.set_content(R"({"error":{"message":"model is loading"}})", "application/json"); return; } relay(backends[0].port, q, r); Incomplete Device Support Check The device support check in line 532 uses device == "vulkan+rocm" which might not correctly identify all devices that should support these routes. This could lead to incorrect behavior if the device string is formatted differently. const bool llama = device == "vulkan" || device == "hrx" || device == "rocm" || device == "vulkan+rocm";

[github-actions]

Persistent review updated to latest commit 631c08c