Eval bug: QWEN3.8:27b + lemonade server's rocm b10472 + cline (vscode) = tools partially working (MCP failures), while vulkan works flawlessly

[MetalKnight]

Name and Version

lemonade server 11.7.0 + rocm b10472

Operating systems

Windows

GGML backends

HIP

Hardware

AMD AI MAX 395+ 64 GB (gfx1151)

Models

Qwen3.6:27b and 35b-3ba, Qwen3.8:27b

Problem description & steps to reproduce

vscode + cline 4.1.x + lemonade server + rocm

Current status:

• Tool calling works partially with Qwen3.8:27b:
  native Cline tools (read_files, search_codebase, editor, etc.) are correctly
  recognized and invoked.
• MCP tools still fail: when MCP servers are connected in Cline and their tools
  are injected into the context, the model does not correctly invoke them on the
  ROCm backend. The same setup works correctly on the Vulkan backend.

model is loaded on both backends with

--flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --no-mmap --temp 0.1 --repeat-penalty 1.0 --top-k 20 --top-p 0.95 --min-p 0 --chat-template-kwargs '{"preserve_thinking":true,"reasoning_effort":"low"}'

Hypothesis: the issue may be context-size related — MCP tool schemas significantly
increase the prompt token count, and there may be a ROCm-specific behavior difference
under high context load compared to Vulkan.

First Bad Commit

No response

Relevant log output

I don't have any, let me know if lemonade server logs are enough, I can provide them.

[MetalKnight]

relates to
lemonade-sdk/lemonade#2977

[nabe2030]

This may be the same underlying defect discussed in
#27579 (comment) and its follow-up
(#27579 (comment)) — a process_ubatch
input-write race on HIP, amplified by the APU host-buffer path from #24233, which we found
causes tool-call corruption specifically at longer context / larger tool arrays even on a single
slot. Your MCP-tools-fail-but-native-tools-work pattern and context-size hypothesis line up with
what we saw.

If you're able to test against a build with the sync fix from #27572 applied (or the
integrated=false workaround as a quicker stopgap), that would help confirm or rule out the
connection.

[MetalKnight]

@nabe2030 thanks for the reply!
I would gladly test the PR #27705 however I don't have the ROCM SDK to rebuild lemonade-server.exe with HIP ROCM support.
Is there any artifact I could use in the PR CI?
Otherwise I can test once https://github.com/lemonade-sdk/llama.cpp ports the fix in its fork.
or I don't if someone from lemonade's team like @superm1 could help here triggering a nightly with that fix... :)

[nabe2030]

@MetalKnight — apologies, I pointed you in a vague direction and you followed it to the
wrong PR. #27705 is a different defect (an output_reorder index-space mismatch on
the MTP/nextn path); it won't touch what you're seeing.

The one that matters for your symptom is #27311 (pwilkin, "Scheduler UMA ring
buffer"), or #25863 as a smaller targeted mitigation of the same thing. Both address
the write-after-read on graph input tensors that the HIP host-buffer path exposes on APUs.

On the CI artifact question: I don't think that route works. Looking at #27311's check
list, there's no Windows ROCm job — the ROCm Windows zips come from the release workflow
on master, not from PR CI. So you'd be waiting for the lemonade fork either way.

But you can probably test this today without rebuilding anything. @Superfish1000
measured the trigger precisely in #27311 on gfx1151: corruption appears whenever
n_ubatch < n_batch, and for ordinary generation whenever prompt_tokens > n_ubatch.
The only passing configuration is the one where the whole prompt lands in a single
ubatch — 7/7 in his generation tests, 8/8 on the batch-split rule.

That is a runtime flag, not a code change. If lemonade lets you pass through
--batch-size and --ubatch-size, try setting them equal and large enough to hold your
MCP-loaded prompt:

 `--batch-size 8192 --ubatch-size 8192` 

(pick a size above your actual prompt token count; larger ubatch costs compute-buffer
memory, so start from whatever your MCP context actually measures). If your MCP tools
start being invoked correctly at ub == b and break again when you drop -ub back to
512, that both confirms the connection to this defect and gives you something usable
before any fix lands.

Your "native Cline tools work, MCP tools fail" split fits this well: MCP schemas are what
push the prompt past n_ubatch, and the failure mode we documented in #27579 is exactly
that the head of a large tool array goes missing while the tail is read correctly — the
model keeps answering fluently and simply doesn't see the tools that were injected first.

One more thing worth knowing before you spend effort staying on ROCm: @thedelph measured
the backend switch on this chip in #27506 and found Vulkan cost 5–9% on prefill but
gained 24% on decode (49.5 → 61.5 t/s). Since Vulkan already works flawlessly for
you, there may be no throughput argument for waiting at all.

[MetalKnight]

Thanks for the detailed reply,
Now I fully understand the status.
I will try to do some measurements and come back, along with some decoding speeds once the fix is merged in main and propagated to lemonade.