docs(vision): negative result — temperature 0.01 is not a safe substitute for card sampling

[glennneuber]

Records an approach that does not work, so nobody retries it. 9 models × 3 repeats × both think modes, eight contract arms, num_predict 8192 think-on.

The result

A capped cell is eval_count == num_predict — reasoning never terminated.

model	capped of 8, per repeat
gemma4:12b-it-q4_K_M	6, 6, 5
qwen3.6:35b-a3b-q4_K_M	6, 6, 4
gemma4:26b-nvfp4	1, 3, 2
gemma4:26b-a4b-it-q4_K_M	1, 1, 2
gemma4:31b-mxfp8	1, 2, 1
gemma4:26b-mxfp8	1, 1, 1
gemma4:31b-nvfp4	0, 1, 0
gemma4:26b-mlx-bf16	0, 0, 0
gemma4:31b-it-q4_K_M	0, 0, 0

The budget is excluded as a cause. At card sampling with the same 8192 budget, all three of the retested models capped 0 of 8 in the 18-model campaign. The caps are the temperature.

It is not gemma4:12b-specific

An earlier framing would have scoped this to gemma4:12b and pointed at its distinct build — the only gemma4 with a separate 52.38M CLIP projector, audio capability and requires 0.30.5. That tidy story dies on qwen3.6:35b-a3b, which shares none of it and runs away just as hard.

No other structural variable survives either:

• Not size — 31b-mxfp8 caps 1–2 while 31b-it-q4_K_M caps 0
• Not quantisation — q4_K_M spans the whole range, 6 of 8 to 0 of 8
• Not engine — MLX and GGUF both appear at 0 and mid-range

The incidence is bimodal and unexplained, and is recorded as an open question rather than attached to whichever variable happens to correlate — the discipline that killed the MoE hypothesis in the campaign.

What to do instead

Repeats at card sampling, which is what ADR 0022's own supersession did (n=3). SAMPLING=legacy is strictly worse than 0.01, not better.

A useful side result

Think-off at temperature 0 is effectively deterministic — 7 of 9 models byte-identical across three repeats. That retroactively strengthens every think-off result in the 18-model campaign, all of which were n=1, so the axis-flip counts behind SPEC C2 rest on repeatable measurements.

Both models that varied are mxfp8, consistent with temperature 0 not being reproducible across model loads on MLX — but that's 2 models at n=3 and is not promoted to a claim.

Harness trap documented

The first attempt ran vision_suite.py directly and inherited the think-off num_predict of 2200 for think-on cells — ADR 0022 trap #1, which confounds exactly this experiment since a budget cap is indistinguishable from a decoding cap. Those outputs were discarded rather than scored.

gemma4:31b-mlx-bf16 deferred for memory headroom, noted as the most informative missing cell.

🤖 Generated with Claude Code