feat(vision-suite): refuse a think-on arm that cannot climb the ladder

[glennneuber]

The rung is a result, not plumbing

The num_ctx a think-on cell converges at is a throughput fact about the model. KV size drives decode speed — which is exactly why two cells measured at different rungs aren't comparable on tok/s or req/h, and why the scores carry num_ctx at all. "This model needs 32768 to finish a thinking response" is the number that sizes a deployment.

Pinning CTX_MAX to the think-on start rung doesn't make an arm cheaper. It makes that number unobtainable: escalation is impossible, the cell caps, and the window it actually needed is never discovered.

The guard

run_engine_compare.sh exits 2 when CTX_MAX leaves no rung above the think-on start rung.

• ALLOW_NO_LADDER=1 is the deliberate opt-out, and obliges the write-up to say a fixed window was used.
• Think-off is unaffected — it has a fixed cap by design and nothing to climb.
• A lowered ceiling is still fine. The guard blocks no escalation, not a smaller one.

Why refuse instead of warn

Downstream the failure is invisible. A capped cell and a cell that genuinely converged at the start rung both write a num_ctx into the scores, and every summarizer renders them identically. A warning scrolls past in a run that takes hours; a wrong throughput number gets published.

This is a real defect I committed today

I passed a fixed CTX_MAX=16384 to a think-on arm at temperature 0 — the sampling regime where non-termination is the expected failure — so no capped cell could have escalated.

My stated reason was comparability with the 2026-08-17 low-temperature arm, which has no ladder because it was a hand-written loop (ADR 0028). Its own text says so:

There is no ladder in this arm: it was run directly rather than through run_engine_compare.sh, so every cell sits at a fixed window and a missing score means the model did not finish at that window, not that it failed.

Reproducing a bespoke arm's fixed window inside the sanctioned runner launders its defect into the one place that defect was supposed to be impossible. Comparability against an arm that could not measure the rung is not a reason to also not measure it.

That's the gap this closes: ADR 0028 stops you writing a seventh loop, but nothing stopped you passing the runner a configuration that disables the thing the runner exists for.

Verification

invocation	result
CTX_MAX=16384 THINK_MODES=on	refuses, exit 2
CTX_MAX=32768 THINK_MODES=on	proceeds
CTX_MAX unset, THINK_MODES=on	proceeds — num_ctx=16384, num_predict=8192
CTX_MAX=16384 THINK_MODES=false	proceeds, unaffected
ALLOW_NO_LADDER=1 + any of the above	proceeds

Docs

SPEC: vision harness reuse gains H4a and a conformance row, alongside the existing H1–H8.

Independent of #168, #171 and #172 — no shared files.

🤖 Generated with Claude Code