Repository navigation
preflight: gate think+format on both Apple stacks, and add the missing metal 0.32.14 profile - #173
Merged
Merged
Conversation
…g metal 0.32.14 profile The fork's headline feature had NO preflight gate on the platform it is actually served from. ADR 0004 closes the /api/generate side of think+format that upstream left open (ollama#12460 fixed only /api/chat in v0.12.4), and ADR 0009 gives the MLX runner the constrained sampling it did not have — and neither Apple profile had a think_format block, so the check skipped four times per run with "no think+format expectation recorded". Measured 2026-08-17 on the native macOS serve, server 0.32.14-maxusai-9594f81e. All six cells emit valid JSON after non-empty thinking: metal gemma4 eval 238 thinking 774ch body 211ch metal qwen35moe eval 1044 thinking 3543ch body 349ch mlx-metal gemma4 eval 116 thinking 292ch body 174ch mlx-metal gemma4_unified eval 796 thinking 2859ch body 241ch mlx-metal qwen35moe eval 966 thinking 3664ch body 322ch mlx-metal qwen35 eval 297 thinking 858ch body 373ch None capped. The probe samples greedy and the check's own diagnosis warns that can stop reasoning terminating; the worst cell used 1044 of 4000, so the budget carries ~4x headroom and an empty response here means the fix regressed rather than the budget being short. require_nonempty_thinking is asserted because all six produce it — a silent loss of thinking is exactly the regression worth catching. ALSO FIXES A GAP FROM THE UPSTREAM MERGE. The payload bump got mlx-metal-0-32-14 but the GGUF/Metal stack was left behind, so --platform metal exited 2 on the current build while mlx-metal resolved — same binary, same host, one stack gated and the other not. Adds metal-0-32-14 with ladders RE-MEASURED on b10434 rather than carried over: gemma4 flat 1102, qwen35moe [1034,1034,1034,2306,4082]. Both reproduce their 0.32.5 values exactly, which is the result rather than the method. The 1034 floor is llama_server's qwen-VL --image-min-tokens policy, not the model — the MLX sibling of those same weights ladders from 68. A first attempt wrote these blocks onto the 0.32.5-pinned `metal` and `mlx-metal` profiles while measuring on 0.32.14 — the ADR 0011 violation this whole mechanism exists to prevent. Reverted and redone against the 0.32.14 profiles; the preflight SKIP is what caught it. Verified: verdict gate 34/34; --platform mlx-metal now 14 PASS (was 10, +4 think_format); --platform metal resolves at all for the first time on this build, 8 PASS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…wn form The six blocks added in the previous commit invented a comment shape -- "# measured: eval_count N, thinking Nch, body Nch" -- instead of using the form cuda-dynres-903.qwen35 already established: a prose sentence giving eval_count against the allowance and the thinking length, followed by whether the floor is load-bearing. Same failure as the invented chat table: a format nobody else uses and nobody asked for. Now matching, and dropping the body-length figure the convention does not carry. Also states something the previous comments implied and should not have: the token counts are INDICATIVE ONLY. Think-on is stochastic, and a preflight re-run of these same cells measured different figures (gemma4_unified 796 -> 1277, qwen35moe 966 -> 1439). The check asserts "valid JSON after thinking", not a token count, and the comment now says so rather than presenting a sampled value as a stable property. Verified: verdict gate 34/34 after the edit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The fork's headline feature had no preflight gate on the platform it's actually served from. ADR 0004 closes the
/api/generateside of think+format that upstream left open (#12460 fixed only/api/chat, v0.12.4), and ADR 0009 gives the MLX runner constrained sampling it didn't have — yet neither Apple profile had athink_formatblock, so the check skipped four times per run.Measured, all six cells pass
Server
0.32.14-maxusai-9594f81e, native macOS serve:All valid JSON after non-empty thinking. None capped — the probe samples greedy and the check's own diagnosis warns that can stop reasoning terminating, but the worst cell used 1044 of 4000, so an empty response here means the fix regressed rather than the budget being short.
require_nonempty_thinkingis asserted because all six produce it — a silent loss of thinking is exactly the regression worth catching.Also fixes a gap the upstream merge left
The payload bump created
mlx-metal-0-32-14but left the GGUF/Metal stack behind:--platform metalexited 2 on the current build whilemlx-metalresolved. Same binary, same host, one stack gated and the other not.metal-0-32-14adds ladders re-measured on b10434, not carried over — gemma4 flat1102, qwen35moe[1034,1034,1034,2306,4082]. Both reproduce their 0.32.5 values exactly, which is the result rather than the method. The1034floor is llama-server's qwen-VL--image-min-tokenspolicy, not the model: the MLX sibling of those same weights ladders from 68.One error caught by the harness
A first attempt wrote these blocks onto the 0.32.5-pinned
metalandmlx-metalprofiles while measuring on 0.32.14 — the exact ADR 0011 violation the mechanism exists to prevent. The preflightSKIPis what surfaced it; reverted and redone against the 0.32.14 profiles.Verification
🤖 Generated with Claude Code