[leehack]

Summary

Builds chat-template messages the way llama.cpp's common_chat_msg::to_json_oaicompat does at the pinned 7fe450e19, in one shared path (messagesForTemplate) that every handler now uses.

• #715: an assistant turn with only tool calls or only reasoning got content: null; it now gets "", as llama.cpp does, including on the image (typed-content) path. Only a template that reads content solely as parts (supports_typed_content without supports_string_content) gets one empty text part instead, as llama.cpp's input normalizer does. At base, QwQ-32B threw (Call to non-function: Undefined), LFM2-8B printed null, and Devstral printed <function text>.
• #717: the LFM2, gpt-oss, Solar Open, Ministral, DeepSeek V3 and TranslateGemma handlers built messages with splitToolResults(...).map(toJson), which skipped the typed-result conversion. Map and List results reached those templates raw. All six now go through templateMessages, and FunctionGemma now renders its non-tool messages through it too.
• #702: earlier tool-call arguments stayed a JSON string. llama.cpp decodes them into an object when the template reads them as one (supports_object_arguments, gated since ggml-org/llama.cpp#20612).
  • JinjaAnalyzer now runs llama.cpp's own object-arguments probe conversation and sets TemplateCaps.supportsObjectArguments.
  • When that cap is set, messagesForTemplate decodes JSON-object arguments. Other arguments stay strings.
  • Handlers that already normalize arguments are unchanged.

Why none was the default: none and normalizeOnly come from the per-format func_args_not_string port in #38, kept by #158. Upstream later replaced that with the caps check. No doc, test or commit gives a prompt-cache reason for keeping none.

Why #702 cannot hurt prompt reuse: native ctx.cachedPromptTokens (llama_cpp_service.dart) and llama-web-bridge's g_cached_prompt_tokens hold prompt tokens only, never generated tokens. Measured on qwen2.5-0.5b over a 3-turn tool conversation:

Turn	Cached tokens	Equals
2	172	the full turn-1 prompt
3	213 (base) / 212 (head)	the full turn-2 prompt

Production-readiness scope

• User-facing scope: Chat prompts for tool conversations match llama-server for tool-call-only and reasoning-only turns, typed tool results in the six handlers, and earlier tool-call arguments.
• Supported platforms/paths: All. The change is in the shared Dart template path (native, WebGPU, LiteRT-LM templates).
• Unsupported or intentionally unavailable paths: Malformed or non-object arguments stay strings. llama-server rejects such a request; llamadart keeps rendering it as before.
• Out of scope / follow-ups: These gaps exist at base and are unchanged here:
  • llama.cpp's specialized message edits for gpt-oss, Ministral and LFM2 (thinking, content blocks);
  • the LFM2.5 tool-list shape (#716);
  • #720: llamadart's string/typed content caps differ from llama.cpp's /props for SmolVLM, TranslateGemma 2B, Ministral 3 and LFM2.5. It covers:
    • SmolVLM text-only chats losing all message text;
    • the Ministral 3 image plus reasoning-only throw;
    • the Kimi-K2 image-path tool-result list, which depends on that fix.
  • Apriel 1.6 uses llamadart's generic tool format, not llama.cpp's.

Completeness checklist

• Declared scope is fully implemented, or explicitly reduced above.
• Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable.
• Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant. There is no public API change; the changelog is updated in both files.
• Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
• Security/privacy review completed: no secrets, bearer tokens, signed URLs, or raw secret-bearing paths leak through logs, cache keys, metadata, errors, or snapshots.
• Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above.

PR type guidance

• Bugfix PR: Root causes are above. Regression tests:
  • template_message_json_parity_test.dart holds 21 llama-server /apply-template prompts (test/fixtures/template_message_json_upstream.{json,md}). Covered families:
    • Tool-call-only assistant turns reach templates with content null; llama-server passes "" #715: Devstral, LFM2 and QwQ; with an image and a tool-call-only turn (multimodal server, LFM2-VL-450M + mmproj): Gemma 4, Kimi-K2 and Apriel 1.6 on generic routing;
    • Six template handlers pass Map/List tool results raw, bypassing messagesForTemplate normalization #717: Map and List results for Ministral, DeepSeek V3.1, Solar Open, LFM2 and gpt-oss;
    • Earlier tool-call arguments reach templates as a string, so prompts differ from llama-server #702: Qwen3, Ministral and gpt-oss arguments;
    • Functionary v3.2 as a control (caps false).
  • Per-format argument cases and a typed-only template case in chat_template_handler_test.dart.
  • Handler tests for TranslateGemma and FunctionGemma.
  • Caps-probe cases in jinja_analyzer_test.dart.
  • Platforms: all.
• Feature PR: N/A, bugfix.
• Docs-only PR: N/A.

Test Plan

At head ad8dd330d, run serially on a loaded shared Mac.

• dart run tool/prepare_workspace.dart: no tracked changes, at head and base.
• dart format --output=none --set-exit-if-changed .
• dart analyze
• dart run tool/testing/check_platform_boundaries.dart
• dart test -p vm -j 1 --exclude-tags local-only: 3,498 pass. No test was removed compared with base eeb382b55; 79 were added.
• dart test -p chrome --exclude-tags local-only: 1,655 pass. No test was removed; 56 were added.
• Other targeted/local-only validation:
  • ./tool/testing/run_template_parity_suites.sh passes. Its unit template suite and compiled grammar row run with the llama.cpp validator.
  • (cd example/chat_app && flutter test) passes: 491 tests.
  • flutter test --platform chrome test/chat_generation_service_test.dart passes.

Differential against llama-server 7fe450e19 /apply-template, base eeb382b55 vs head ad8dd330d:

• Templates: 101 (test/fixtures, tool/litert_lm_templates, llama.cpp models/templates).
• Conversations: 12 text (tool-call-only, reasoning-only, parallel calls, string/Map/List results, spaced/nested/unicode arguments) and 6 with an image (tool-call-only, reasoning-only, parallel typed results, text and tool call, text only). Each runs with tools on and off: 3,636 renders per side.
• Server: text on stories15M; image conversations on LFM2-VL-450M with its mmproj.
• Normalized before comparing: each random <__media_…__> marker, and LFM2-VL's <|im_end|> EOS, compared as the harness's </s>.
• Moves are judged per changed hunk and by edit distance.

Result	Text	Image
Changed, now identical to the server	682	207
Changed, closer to the server	73	123
Newly render, identical to the server	99	34
Changed away from the server	0	0 by hunk; 1 by edit distance, see below
Newly throw	0	0
Newly render (base threw), still differ from the server	29	28
No server counterpart (LiteRT built-ins)	4	4
Unchanged	1,537	815

The one image render: RWKV World with tools, image, a tool call, then a reasoning-only turn. Its only changed hunk inserts Assistant: \n\n for the reasoning-only turn, exactly as llama-server renders it.

• Head inserts Assistant: \n\n for the reasoning-only turn, exactly as llama-server renders it. Without tools, the head render equals the server's except for the <think> generation prompt llama-server adds, a gap that exists at base.
• The edit distance still rises from 271 to 279, because the generic tool JSON already in the prompt (not in the server's) dominates.

The previous head 9cbf8e140 moved 26 image renders away (Gemma 4, Kimi-K2, Apriel 1.6, TranslateGemma 2B, RWKV). This head fixes 25 of them; the 26th is the RWKV render above.

Detected supports_object_arguments equals llama-server /props on 97 of 97 templates.

Controls:

• Base. With the head tests copied onto base eeb382b55:

  • 17 of 18 parity cases fail. The Functionary control passes.
  • 24 of 43 per-format argument cases fail.
  • The TranslateGemma typed-result case and the Ministral fixture case fail.

• Previous head 9cbf8e140. With the new fixture and test copied in, the 3 image cases fail: Gemma 4 closes the model turn after the tool response, Kimi-K2 prints [{'type': 'text', 'text': ''}], and Apriel 1.6 loses the tool call.

• Mutations, each catching its fix:

  Mutation	Tests that fail
  Keep null content	10
  Empty content as a text part on the image path (the previous head)	5, including the 3 image cases
  Typed-only flag ignored	3
  Typed-only flag not passed from TemplateCaps	1
  objectArguments off	34
  Probe counts a whole-object print	5
  Probe adds the tool name	3
  Each of those seven handlers (the six plus FunctionGemma) restored to base	2–6
  Each of the 33 handler files with its templateMessages call bypassed	Its own format case (granite: every handler renders one tool message per tool result)

• Parser and grammar. Negating the parallel-calls cap fails the Ministral and Nemotron parser-bound tests. Negating the typed-content cap fails the audio rollback and compiled-grammar e2e cases.

Matrix Evidence

Matrix row	Scope covered	Platform / model / backend	Result	Evidence / notes
root-vm / root-chrome	full root suites	macOS VM, Chrome	PASS	3,498 / 1,655 pass at ad8dd330d
structured-output-adversarial	input rendering/history, compiled grammar, schema, rollback, tool choice/thinking, upstream parity	Dart template engine; llama-server 7fe450e19 fixtures	PASS	run_template_parity_suites.sh; differential, controls and mutations above
examples-tests	chat_app	VM, Chrome	PASS	491 tests; the Chrome generation service test passes
high-risk-exact-head-independent-qa	independent blocking-only review of the exact head against the current base	fresh auditor claude-opus-5-5-fresh-subagent-audit719	PASS	Accepted at ad8dd330d: 0 PR-caused P1s, 0 unresolved threads (record below)
release-metadata-verification	—	—	N/A	No release metadata change

Review Notes

• Independent review status: accepted at ad8dd330d by claude-opus-5-5-fresh-subagent-audit719; round 1 at 9cbf8e140 was rejected for a P1, fixed here.
• CI status / head SHA: 23 checks pass, 5 skipped, on ad8dd330de1d129013cc67c2ce4b3978073e5ea6.

High-risk regression review

• Classification: high-risk (structuredOutput), from classify_high_risk_changes.dart.
• Implementation task: this PR's author session.
• Independent blocking QA task: accepted. Record:

{"auditor_identity":"claude-opus-5-5-fresh-subagent-audit719","audit_kind":"operator-owned","audit_head_sha":"ad8dd330de1d129013cc67c2ce4b3978073e5ea6","audit_base_sha":"eeb382b552b5eaecba39338f7a540f673ec3e59e","decision":"accepted","unresolved_review_threads":0,"known_pr_caused_p1_regressions":0,"summary":"2026-09-25 round 1 (9cbf8e14): rejected for a P1: multimodal null content became [{'type':'text','text':''}], so Gemma 4 31B/LiteRT gemma4 closed the model turn after a tool response, Apriel-1.6 dropped the tool call, and Kimi-K2/TranslateGemma printed the list repr. Round 2 (ad8dd330): null becomes '' unless the template is typed-only by TemplateCaps. All 5 repros fixed (4 byte-identical to base, Kimi-K2 only toward-server arg spacing). Own differential, 101 templates x 28 conversations vs llama-server 7fe450e19 (text + multimodal) plus 8 local GGUF templates: 0 new throws, non-prompt fields unchanged, 2 away (RWKV World) whose single hunk inserts 'Assistant: \\n\\n' exactly as llama-server does, with the distance rise from pre-existing generic tool JSON. Typed-only rule flips no template; the SmolVLM/TranslateGemma 2B/Ministral 3/LFM2.5 caps mismatches produce nothing worse than base. Ministral-3-14B image+reasoning throw and TranslateGemma 2B renders identical to base. Object-args probe matches /props on 101/101. Mutations red; all gates green; CI green; 0 threads."}

• Exact head / current base: ad8dd330de1d129013cc67c2ce4b3978073e5ea6 / eeb382b552b5eaecba39338f7a540f673ec3e59e.
• Production-branch deletion, bypass, or miswire proof: Each of the 33 handler files, with its templateMessages call bypassed, fails a test. Each core change (content, the typed-only part and its wiring, objectArguments, the probe, the probe's tool message) fails tests when mutated.
• Affected-family real-model/artifact evidence: None; no local weights for these families.
• Explicit unavailable-family or other N/A evidence:
  • Devstral, LFM2, QwQ, Ministral 3, DeepSeek V3.1, Solar Open, gpt-oss, Qwen3 and Functionary v3.2 are pinned to llama-server 7fe450e19 emissions of their templates.
  • Gemma 4, Kimi-K2 and Apriel 1.6 are pinned to multimodal llama-server 7fe450e19 emissions.
  • TranslateGemma: llama-server did not start with that template, so an inline-template handler test covers it.
  • FunctionGemma: llama-server reports caps false, and llamadart detects the same.
• Evidence: with this audit record as independent_audit and the QA row passing, high_risk_readiness.dart exits 2 (unverifiedPrerequisites). The local evidence is consistent; only the external prerequisites are unverified.
• Known PR-caused P1 regressions: 0
• Unresolved review threads: 0

[github-actions]

Chat app preview removed for leehack/llamadart-chat-pr-719.