From 78662209c3d453f387d9f41e543401ec43afe0f3 Mon Sep 17 00:00:00 2001 From: Glenn Neuber Date: Sun, 27 Sep 2026 23:55:46 +1000 Subject: [PATCH 1/3] docs(fold): v0.34.4-dynres is tagged; Metal's final 26b count in the CUDA section - Status: tag and deploy. v0.34.4-dynres is cut on b43ee8e37, the #375 merge (ADR 0032), so the release build stamps 0.34.4-dynres-0-gb43ee8e, which cuda-dynres-903's version_pattern already accepts. The tag push started release.yaml, which stopped at its first step as designed (no self-hosted runners). The CUDA deploy stays prepared, not run. - The CUDA think-on section still quoted Metal's interim gemma4:26b count ("five more"); Metal's final count is 4 against 1 (#375). #389 fixed the gfx1151 section's copy; this fixes the other. Co-Authored-By: Claude Opus 5.5 --- docs/maxusai/tasks/upstream-sync-0.34.4.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/maxusai/tasks/upstream-sync-0.34.4.md b/docs/maxusai/tasks/upstream-sync-0.34.4.md index 41d1aac75ef..8b8bd9a92ae 100644 --- a/docs/maxusai/tasks/upstream-sync-0.34.4.md +++ b/docs/maxusai/tasks/upstream-sync-0.34.4.md @@ -17,7 +17,7 @@ lands on this branch, your gate 4 and gate 6 legs can build from it. | 4, image | **done on CUDA** — `e8f7a2a1968c`, a full build. MLX tests on its payload 889 passed, 0 failed; the vision goldens identical to 0.34.2's; **done on gfx1151**: `0.34.3-dynres-5-g29ae523-rocm7-gfx1151`, with a b11081 payload whose structure is unchanged against 0.34.3's; rebuilt with 908 as `0.34.3-dynres-22-g5584539`, and **908 changes no gfx1151 kernel** | | 5, preflight | **PASS on CUDA**: run 2 PASS=21 SKIP=8, with the pins moved in `c79e50d98` after run 1 (FAIL=2 on the two pins, by design); run 3, on the image with 908, the same. **gfx1151: PASS=20 SKIP=12** with the new `rocm7-0-34-4-dynres` (#378), and the same on the 908 image | | 6, campaigns | **done on CUDA** (2026-09-27). GGUF think-off: 210 of 6,909 cells move, in the head-dimension-256 models only, and reverting `ce8caa6e6` restores production on both probes: its device half on gemma4:31b, its host half on qwen3.6. On the image with 908, gemma4:31b, 26b and e4b and nemotron3 equal production; e2b and the qwen models keep the host half's movement. MLX think-off: no consistent difference, inside or across MLX-CUDA's run-to-run spread. OCRBench: production's scores, but for one reproducible item on GGUF q4, which is `ce8caa6e6`'s device half. Think-on: `ce8caa6e6`'s device half leaves 6 of 27 gemma4:26b cases in loops that never end, against 1 without it; 31b is unaffected. MLX think-on: the single pass loops no more than two-pass, with or without a fixed history. gemma4:26b `multi_3img_anchored` never converges in any of 8 runs. Item 8: its trap sentence alone loops it on GGUF on the fold image, not on the image that ships; on MLX it finishes about one cold draw in five, with the sentence or without it. Drafting: under production's knob the single pass never drafts, so it thinks 1.5–1.7× slower, and the deploy runs two-pass (open item 7). See [Gates 4–6 on CUDA](#gates-46-on-cuda-2026-09-25). **gfx1151:** think-off and OCRBench equal production in every scored cell. Think-on under the aligned protocol is done (2026-09-27): on all five GGUF models the single pass leaves the same cases unfinished as two-pass, and those loops come from the prompt (#387). See [Gates 4–6 on gfx1151](#gates-46-on-gfx1151-2026-09-25) | -| tag and deploy | **not yet.** The CUDA deploy is prepared. It mirrors production (`0.34.2-dynres-0-g5bffaac`) and adds `OLLAMA_KV_CACHE_TYPE=f16` and `OLLAMA_FORMAT_TWO_PASS=1` (open items 6 and 7). It waits on the merge, the `v0.34.4-dynres` tag, the release image and the maintainer's word | +| tag and deploy | **tagged** `v0.34.4-dynres` on `b43ee8e37`, the #375 merge (2026-09-27; ADR 0032), so a release build stamps `0.34.4-dynres-0-gb43ee8e`, which `cuda-dynres-903`'s `version_pattern` already accepts. The tag push started `release.yaml`, which stopped at its first step as designed: this fork has no self-hosted runners. **The CUDA deploy is prepared, not run.** It mirrors production (`0.34.2-dynres-0-g5bffaac`) and adds `OLLAMA_KV_CACHE_TYPE=f16` and `OLLAMA_FORMAT_TWO_PASS=1` (open items 6 and 7). It waits on the release image, gated like 0.34.2's, and the maintainer's word | **Three hosts converged on this merge.** The ROCm and Metal hosts had each started the same fold before #375 existed, stopped, and cross-checked instead; see [#375](https://github.com/MaxusAI/ollama/pull/375). @@ -433,9 +433,9 @@ apply. The other hosts' loops are different cases. The MLX single-pass loop the Metal host found on gemma4:31b-nvfp4 does not reproduce on MLX-CUDA: no loop in four runs, and here the spread within one flow is as large as the spread between -flows. On gemma4:26b-nvfp4, Metal's single pass leaves five more cases looping than its two-pass (interim), and Metal's -discriminator puts that on drafting rather than on the flow. gfx1151 has one pre-existing gemma4:26b GGUF loop, -`bbox_contract_real_1img`, byte-identical on b10969. On CUDA, the loop-rate run covers ROCm's case, and the MLX +flows. On gemma4:26b-nvfp4, Metal's single pass leaves 4 cases unfinished against two-pass's 1 (its final count, #375), +and Metal's discriminator puts that on drafting rather than on the flow. gfx1151 has one pre-existing gemma4:26b GGUF +loop, `bbox_contract_real_1img`, byte-identical on b10969. On CUDA, the loop-rate run covers ROCm's case, and the MLX think-on run covers Metal's: the two cases Metal found on 31b, on all four models, plus Metal's five 26b cases on gemma4:26b-nvfp4, both flows, two repeats each, the full ladder. From 7673dd546f4f5ad6e1fb5cbcdd1a9fe0d6f5d79a Mon Sep 17 00:00:00 2001 From: Glenn Neuber Date: Mon, 28 Sep 2026 00:12:16 +1000 Subject: [PATCH 2/3] docs(fold): the CUDA release image is built and gated maxusai/ollama:sync-0.34.4-main (bf59c2eab9b8) is the gated sync-0.34.4-908 with a Go-only swap to the 0.34.4-dynres-0-gb43ee8e binary, as 0.34.2's release was. Nothing a native stage copies changed since 5584539 but llama/compat/README.md; all 2,696 payload files are hash-identical to -908's. Preflight on a canary from the tag's harness: VERDICT PASS, PASS=21 SKIP=8, every check's status equal to run 3's on -908. The deploy waits on the maintainer's word. Co-Authored-By: Claude Opus 5.5 --- docs/maxusai/tasks/upstream-sync-0.34.4.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/maxusai/tasks/upstream-sync-0.34.4.md b/docs/maxusai/tasks/upstream-sync-0.34.4.md index 8b8bd9a92ae..a9aff645270 100644 --- a/docs/maxusai/tasks/upstream-sync-0.34.4.md +++ b/docs/maxusai/tasks/upstream-sync-0.34.4.md @@ -17,7 +17,7 @@ lands on this branch, your gate 4 and gate 6 legs can build from it. | 4, image | **done on CUDA** — `e8f7a2a1968c`, a full build. MLX tests on its payload 889 passed, 0 failed; the vision goldens identical to 0.34.2's; **done on gfx1151**: `0.34.3-dynres-5-g29ae523-rocm7-gfx1151`, with a b11081 payload whose structure is unchanged against 0.34.3's; rebuilt with 908 as `0.34.3-dynres-22-g5584539`, and **908 changes no gfx1151 kernel** | | 5, preflight | **PASS on CUDA**: run 2 PASS=21 SKIP=8, with the pins moved in `c79e50d98` after run 1 (FAIL=2 on the two pins, by design); run 3, on the image with 908, the same. **gfx1151: PASS=20 SKIP=12** with the new `rocm7-0-34-4-dynres` (#378), and the same on the 908 image | | 6, campaigns | **done on CUDA** (2026-09-27). GGUF think-off: 210 of 6,909 cells move, in the head-dimension-256 models only, and reverting `ce8caa6e6` restores production on both probes: its device half on gemma4:31b, its host half on qwen3.6. On the image with 908, gemma4:31b, 26b and e4b and nemotron3 equal production; e2b and the qwen models keep the host half's movement. MLX think-off: no consistent difference, inside or across MLX-CUDA's run-to-run spread. OCRBench: production's scores, but for one reproducible item on GGUF q4, which is `ce8caa6e6`'s device half. Think-on: `ce8caa6e6`'s device half leaves 6 of 27 gemma4:26b cases in loops that never end, against 1 without it; 31b is unaffected. MLX think-on: the single pass loops no more than two-pass, with or without a fixed history. gemma4:26b `multi_3img_anchored` never converges in any of 8 runs. Item 8: its trap sentence alone loops it on GGUF on the fold image, not on the image that ships; on MLX it finishes about one cold draw in five, with the sentence or without it. Drafting: under production's knob the single pass never drafts, so it thinks 1.5–1.7× slower, and the deploy runs two-pass (open item 7). See [Gates 4–6 on CUDA](#gates-46-on-cuda-2026-09-25). **gfx1151:** think-off and OCRBench equal production in every scored cell. Think-on under the aligned protocol is done (2026-09-27): on all five GGUF models the single pass leaves the same cases unfinished as two-pass, and those loops come from the prompt (#387). See [Gates 4–6 on gfx1151](#gates-46-on-gfx1151-2026-09-25) | -| tag and deploy | **tagged** `v0.34.4-dynres` on `b43ee8e37`, the #375 merge (2026-09-27; ADR 0032), so a release build stamps `0.34.4-dynres-0-gb43ee8e`, which `cuda-dynres-903`'s `version_pattern` already accepts. The tag push started `release.yaml`, which stopped at its first step as designed: this fork has no self-hosted runners. **The CUDA deploy is prepared, not run.** It mirrors production (`0.34.2-dynres-0-g5bffaac`) and adds `OLLAMA_KV_CACHE_TYPE=f16` and `OLLAMA_FORMAT_TWO_PASS=1` (open items 6 and 7). It waits on the release image, gated like 0.34.2's, and the maintainer's word | +| tag and deploy | **tagged** `v0.34.4-dynres` on `b43ee8e37`, the #375 merge (2026-09-27; ADR 0032), so a release build stamps `0.34.4-dynres-0-gb43ee8e`, which `cuda-dynres-903`'s `version_pattern` already accepts. The tag push started `release.yaml`, which stopped at its first step as designed: this fork has no self-hosted runners. **The CUDA release image is built and gated** (2026-09-28), like 0.34.2's: `maxusai/ollama:sync-0.34.4-main` (`bf59c2eab9b8`) is the gated `sync-0.34.4-908` with a Go-only swap to the `0.34.4-dynres-0-gb43ee8e` binary. Since `5584539` nothing a native stage copies has changed but `llama/compat/README.md`, and all 2,696 payload files under `/usr/lib/ollama` are hash-identical to `-908`'s. Preflight on a canary, from the tag's own harness: VERDICT PASS, PASS=21 SKIP=8, every check's status equal to run 3's on `-908`. **The CUDA deploy is prepared, not run.** It mirrors production (`0.34.2-dynres-0-g5bffaac`) and adds `OLLAMA_KV_CACHE_TYPE=f16` and `OLLAMA_FORMAT_TWO_PASS=1` (open items 6 and 7). It waits on the maintainer's word | **Three hosts converged on this merge.** The ROCm and Metal hosts had each started the same fold before #375 existed, stopped, and cross-checked instead; see [#375](https://github.com/MaxusAI/ollama/pull/375). From 681253fc6415556e7d5b72a3070db8d563f09f57 Mon Sep 17 00:00:00 2001 From: Glenn Neuber Date: Mon, 28 Sep 2026 07:57:47 +1000 Subject: [PATCH 3/3] =?UTF-8?q?docs:=20v0.34.4=20is=20deployed=20on=20CUDA?= =?UTF-8?q?=20=E2=80=94=20README,=20fold=20record,=20and=20the=20productio?= =?UTF-8?q?n=20preflight=20run?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - :11497 runs ollama-0.34.4-dynres-0-gb43ee8e (sync-0.34.4-main) since 2026-09-28 07:37:54, on the maintainer's word: 13 s without service, 55 models either side, OLLAMA_KV_CACHE_TYPE=f16 and OLLAMA_FORMAT_TWO_PASS=1 added to the mirrored environment. Post-deploy preflight on production: VERDICT PASS, PASS=21 SKIP=8, every check's status equal to the canary's; all 9 KV allocations f16; the startup config reads OLLAMA_FORMAT_TWO_PASS:true. The 0.34.2 container is kept for rollback. - The run is committed (runs/preflight-cuda-0344-prod-gb43ee8e.json, with git add -f: runs/ is ignored), as the README's generator comment asked, so the release matrix now regenerates both production rows: release_matrix.py --version 0.34.4-dynres runs/*.json (cuda, and rocm7 from #391). - README: the current fold is v0.34.4-dynres; the Deployed block names the 0.34.4 build on CUDA and gfx1151 with its two variables; the obsolete 0.34.1 gfx1151 paragraph (all three hosts on one commit, patch 906, retired in 0.34.2) is removed, its history is in amd-upgrade-gate.md. Co-Authored-By: Claude Opus 5.5 --- README.md | 85 ++-- docs/maxusai/tasks/upstream-sync-0.34.4.md | 8 +- .../preflight-cuda-0344-prod-gb43ee8e.json | 471 ++++++++++++++++++ 3 files changed, 511 insertions(+), 53 deletions(-) create mode 100644 docs/maxusai/vision-suite/preflight/runs/preflight-cuda-0344-prod-gb43ee8e.json diff --git a/README.md b/README.md index 25dfc784c6c..e42af1e0615 100644 --- a/README.md +++ b/README.md @@ -23,28 +23,23 @@ > - An experimental MLX runtime for Apple Silicon and CUDA — see the caveats > below before using it for anything that matters. > -> **Current fold:** [`v0.34.1-dynres`](https://github.com/MaxusAI/ollama/releases/tag/v0.34.1-dynres) -> — upstream v0.34.1, llama.cpp `b10864`, MLX `d9add9d1`. `main` moves ahead of this between -> folds; the tag is the fixed point to build and roll back to. -> **Deployed:** `main` at `16649e8`, stamped `0.34.1-dynres-16-g16649e8`, on the CUDA host since -> 2026-09-18 22:30 — the tag's native payload with a Go-only rebuild that adds ADR 0036 (a gemma4 -> vision runner starts from the batch rung holding its image ceiling), with -> `OLLAMA_MLX_DRAFT_UNDER_GRAMMAR=0` on the container (ADR 0033; see the fold's memory finding in -> [`docs/maxusai/tasks/upstream-sync-0.34.1.md`](docs/maxusai/tasks/upstream-sync-0.34.1.md)). -> The matrix below is the tag's full preflight run; the rebuild changes no native input, and its -> verification on production is in the task doc's deploy section. -> On the Apple Silicon host the same commit has served the mlx-metal surface on `:11435` since -> 2026-09-18, stamped `0.34.0-maxusai-8a7ba949` — the same build as `0.34.1-dynres-0-g8a7ba94` (ADR 0032, -> 2026-09-19 amendment) — and was promoted on 2026-09-19 with the MLX #3912 kernel fix kept -> ([ADR 0037](docs/maxusai/adr/0037-keep-the-mlx-3912-kernel-fix.md)), with -> `OLLAMA_MLX_DRAFT_UNDER_GRAMMAR=0` in its launchd environment as on the CUDA container. -> **The AMD/gfx1151 host joined on 2026-09-19 18:55**, stamped `0.34.1-dynres-16649e8c` from a -> full `FLAVOR=rocm` build of the same commit, when the -> [upgrade gate](docs/maxusai/amd-upgrade-gate.md#decision-2026-09-19--the-gate-lifts-on-evidence) -> lifted — it had held that host on 0.32.1 since 2026-07-31, and all three platforms now serve -> one commit. That build **must** carry `llama/compat/906-revert-hip-integrated-flag.patch`: -> llama.cpp b10864 misses upstream's HIP revert by 78 minutes, and without it vision output on -> gfx1151 is silently wrong — no crash, no warning, unchanged token counts. +> **Current fold:** [`v0.34.4-dynres`](https://github.com/MaxusAI/ollama/releases/tag/v0.34.4-dynres) +> — upstream v0.34.4, llama.cpp `b11081` with compat patch 908, MLX `59d600b5`, XGrammar 0.2.7. `main` moves ahead +> of this between folds; the tag is the fixed point to build and roll back to. +> **Deployed:** the tag, stamped `0.34.4-dynres-0-gb43ee8e`. It went onto the CUDA host at 07:37 on 2026-09-28, and +> onto the AMD/gfx1151 host at 07:39 ([#391](https://github.com/MaxusAI/ollama/pull/391)). +> - **Build:** on each host, the natively gated payload with the tag's Go binary. +> - **think+format:** both hosts run the two-pass flow (`OLLAMA_FORMAT_TWO_PASS=1`, ADR 0004; open item 7 of the +> [fold record](docs/maxusai/tasks/upstream-sync-0.34.4.md)). +> - **KV cache:** both set `OLLAMA_KV_CACHE_TYPE=f16` explicitly ([#387](https://github.com/MaxusAI/ollama/pull/387)). +> - **Drafting:** the CUDA container also keeps `OLLAMA_MLX_DRAFT_UNDER_GRAMMAR=0` (ADR 0033). +> +> The matrix below is each host's preflight on its production container. +> On the Apple Silicon host, `0.34.0-maxusai-8a7ba949` has served the mlx-metal surface on `:11435` since +> 2026-09-18. It is the same build as `0.34.1-dynres-0-g8a7ba94` (ADR 0032, 2026-09-19 amendment). It was promoted +> on 2026-09-19 with the MLX #3912 kernel fix kept +> ([ADR 0037](docs/maxusai/adr/0037-keep-the-mlx-3912-kernel-fix.md)), with `OLLAMA_MLX_DRAFT_UNDER_GRAMMAR=0` in its +> launchd environment as on the CUDA container. > Fork builds are stamped `-dynres--g`; `dynres` > names the change that started the fork, not the company that runs it. @@ -54,36 +49,28 @@ ### What tested green for this fold - -| surface | Build identity | Image size ladder | Pinned image budget | thinking on/off | Output quality | fp16 overflow canary | Runner isolation | measured on | -|---|---|---|---|---|---|---|---|---| -| **cuda** | green | green | green | green | skipped | green | green | `0.34.1-dynres-0-g8a7ba94` | -| **mlx-cuda** | not run | not run | not run | not run | not run | not run | not run | — | -| **mlx-metal** | skipped | green | skipped | green | not run | skipped | green | `0.34.0-maxusai-8a7ba949` | -| **apple-silicon-mlx** | not run | not run | not run | not run | not run | not run | not run | — | -| **rocm** | not run | not run | not run | not run | not run | not run | not run | — | -| **cpu** | not run | not run | not run | not run | not run | not run | not run | — | + runs/ is gitignored; a run meant for this table is committed with `git add -f`. + + For v0.34.4 both production runs are committed, and the table below is the + verbatim output of `release_matrix.py --version 0.34.4-dynres runs/*.json`: + runs/preflight-cuda-0344-prod-gb43ee8e.json the CUDA container, after the deploy + runs/preflight-rocm7-0344-prod-gb43ee8e.json the gfx1151 container (#391) --> + +| surface | Build identity | M5 tensor path | Image size ladder | Pinned image budget | thinking on/off | Output quality | fp16 overflow canary | Runner isolation | measured on | +|---|---|---|---|---|---|---|---|---|---| +| **cuda** | green | n/a | green | green | green | not run | green | green | `0.34.4-dynres-0-gb43ee8e` | +| **mlx-cuda** | not run | not run | not run | not run | not run | not run | not run | not run | — | +| **mlx-metal** | not run | not run | not run | not run | not run | not run | not run | not run | — | +| **apple-silicon-mlx** | not run | not run | not run | not run | not run | not run | not run | not run | — | +| **rocm7** | green | n/a | green | green | green | skipped | skipped | green | `0.34.4-dynres-0-gb43ee8e` | +| **rocm10** | not run | not run | not run | not run | not run | not run | not run | not run | — | +| **cpu** | not run | not run | not run | not run | not run | not run | not run | not run | — | Generated by `release_matrix.py` from recorded preflight runs. A surface with no run for this release reads *not run* — absence is shown, never assumed green. A group is reported at its weakest check, so one skipped probe does not read as a pass. diff --git a/docs/maxusai/tasks/upstream-sync-0.34.4.md b/docs/maxusai/tasks/upstream-sync-0.34.4.md index a9aff645270..4ef6330895c 100644 --- a/docs/maxusai/tasks/upstream-sync-0.34.4.md +++ b/docs/maxusai/tasks/upstream-sync-0.34.4.md @@ -17,7 +17,7 @@ lands on this branch, your gate 4 and gate 6 legs can build from it. | 4, image | **done on CUDA** — `e8f7a2a1968c`, a full build. MLX tests on its payload 889 passed, 0 failed; the vision goldens identical to 0.34.2's; **done on gfx1151**: `0.34.3-dynres-5-g29ae523-rocm7-gfx1151`, with a b11081 payload whose structure is unchanged against 0.34.3's; rebuilt with 908 as `0.34.3-dynres-22-g5584539`, and **908 changes no gfx1151 kernel** | | 5, preflight | **PASS on CUDA**: run 2 PASS=21 SKIP=8, with the pins moved in `c79e50d98` after run 1 (FAIL=2 on the two pins, by design); run 3, on the image with 908, the same. **gfx1151: PASS=20 SKIP=12** with the new `rocm7-0-34-4-dynres` (#378), and the same on the 908 image | | 6, campaigns | **done on CUDA** (2026-09-27). GGUF think-off: 210 of 6,909 cells move, in the head-dimension-256 models only, and reverting `ce8caa6e6` restores production on both probes: its device half on gemma4:31b, its host half on qwen3.6. On the image with 908, gemma4:31b, 26b and e4b and nemotron3 equal production; e2b and the qwen models keep the host half's movement. MLX think-off: no consistent difference, inside or across MLX-CUDA's run-to-run spread. OCRBench: production's scores, but for one reproducible item on GGUF q4, which is `ce8caa6e6`'s device half. Think-on: `ce8caa6e6`'s device half leaves 6 of 27 gemma4:26b cases in loops that never end, against 1 without it; 31b is unaffected. MLX think-on: the single pass loops no more than two-pass, with or without a fixed history. gemma4:26b `multi_3img_anchored` never converges in any of 8 runs. Item 8: its trap sentence alone loops it on GGUF on the fold image, not on the image that ships; on MLX it finishes about one cold draw in five, with the sentence or without it. Drafting: under production's knob the single pass never drafts, so it thinks 1.5–1.7× slower, and the deploy runs two-pass (open item 7). See [Gates 4–6 on CUDA](#gates-46-on-cuda-2026-09-25). **gfx1151:** think-off and OCRBench equal production in every scored cell. Think-on under the aligned protocol is done (2026-09-27): on all five GGUF models the single pass leaves the same cases unfinished as two-pass, and those loops come from the prompt (#387). See [Gates 4–6 on gfx1151](#gates-46-on-gfx1151-2026-09-25) | -| tag and deploy | **tagged** `v0.34.4-dynres` on `b43ee8e37`, the #375 merge (2026-09-27; ADR 0032), so a release build stamps `0.34.4-dynres-0-gb43ee8e`, which `cuda-dynres-903`'s `version_pattern` already accepts. The tag push started `release.yaml`, which stopped at its first step as designed: this fork has no self-hosted runners. **The CUDA release image is built and gated** (2026-09-28), like 0.34.2's: `maxusai/ollama:sync-0.34.4-main` (`bf59c2eab9b8`) is the gated `sync-0.34.4-908` with a Go-only swap to the `0.34.4-dynres-0-gb43ee8e` binary. Since `5584539` nothing a native stage copies has changed but `llama/compat/README.md`, and all 2,696 payload files under `/usr/lib/ollama` are hash-identical to `-908`'s. Preflight on a canary, from the tag's own harness: VERDICT PASS, PASS=21 SKIP=8, every check's status equal to run 3's on `-908`. **The CUDA deploy is prepared, not run.** It mirrors production (`0.34.2-dynres-0-g5bffaac`) and adds `OLLAMA_KV_CACHE_TYPE=f16` and `OLLAMA_FORMAT_TWO_PASS=1` (open items 6 and 7). It waits on the maintainer's word | +| tag and deploy | **done on CUDA.** **Tagged** `v0.34.4-dynres` on `b43ee8e37`, the #375 merge (2026-09-27; ADR 0032), so a release build stamps `0.34.4-dynres-0-gb43ee8e`, which `cuda-dynres-903`'s `version_pattern` already accepts. The tag push started `release.yaml`, which stopped at its first step as designed: this fork has no self-hosted runners. **The CUDA release image is built and gated** (2026-09-28), like 0.34.2's: `maxusai/ollama:sync-0.34.4-main` (`bf59c2eab9b8`) is the gated `sync-0.34.4-908` with a Go-only swap to the `0.34.4-dynres-0-gb43ee8e` binary. Since `5584539` nothing a native stage copies has changed but `llama/compat/README.md`, and all 2,696 payload files under `/usr/lib/ollama` are hash-identical to `-908`'s. Preflight on a canary, from the tag's own harness: VERDICT PASS, PASS=21 SKIP=8, every check's status equal to run 3's on `-908`. **Deployed on CUDA at 07:37:54 on 2026-09-28**, on the maintainer's word. `:11497` runs `ollama-0.34.4-dynres-0-gb43ee8e` from `sync-0.34.4-main`, with 13 s without service and 55 models either side. The container mirrors the 0.34.2 one and adds `OLLAMA_KV_CACHE_TYPE=f16` and `OLLAMA_FORMAT_TWO_PASS=1` (open items 6 and 7). Its startup config reads `OLLAMA_FORMAT_TWO_PASS:true`, and all 9 KV allocations in the post-deploy preflight are f16. That preflight ran on production itself: VERDICT PASS, PASS=21 SKIP=8, with every check's status equal to the canary's (`preflight/runs/preflight-cuda-0344-prod-gb43ee8e.json`). The 0.34.2 container is stopped and kept; the rollback is `docker rm -f ollama-0.34.4-dynres-0-gb43ee8e && docker start ollama-0.34.2-dynres-0-g5bffaac`. gfx1151 promoted the same build at 07:39 (#391) | **Three hosts converged on this merge.** The ROCm and Metal hosts had each started the same fold before #375 existed, stopped, and cross-checked instead; see [#375](https://github.com/MaxusAI/ollama/pull/375). @@ -1100,10 +1100,10 @@ representation-sensitive test each, so the fold's attribution stays clean. and #387). Production does not set the variable today; it runs the default, and its log shows f16 in all 12 of its KV cache allocations, so it is not recreated for this alone. The v0.34.4 deploy mirrors production's container by `docker inspect`, as every CUDA deploy has, and adds the variable, and item 7's `OLLAMA_FORMAT_TWO_PASS=1`. If the - live container carries a different value, it refuses rather than choose. The deploy itself waits on the - maintainer's word. + live container carries a different value, it refuses rather than choose. **Deployed 2026-09-28 07:37** with both + variables (status table). 7. **The deploy's think+format configuration on MLX: two-pass** (the maintainer's decision, 2026-09-27, after the - drafting probe under gate 6). The v0.34.4 deploy sets `OLLAMA_FORMAT_TWO_PASS=1` beside item 6's f16. That keeps + drafting probe under gate 6; deployed 2026-09-28). The v0.34.4 deploy sets `OLLAMA_FORMAT_TWO_PASS=1` beside item 6's f16. That keeps today's speed, today's memory behaviour and ADR 0004's flow. Mirroring production alone would have run the single pass with `OLLAMA_MLX_DRAFT_UNDER_GRAMMAR=0`, which is arm F0, and thinks 1.5–1.7× slower than production's two-pass today. `deploy-v0344.sh` adds the switch the way it adds f16: diff --git a/docs/maxusai/vision-suite/preflight/runs/preflight-cuda-0344-prod-gb43ee8e.json b/docs/maxusai/vision-suite/preflight/runs/preflight-cuda-0344-prod-gb43ee8e.json new file mode 100644 index 00000000000..d75119738a0 --- /dev/null +++ b/docs/maxusai/vision-suite/preflight/runs/preflight-cuda-0344-prod-gb43ee8e.json @@ -0,0 +1,471 @@ +{ + "meta": { + "host": "http://127.0.0.1:11497", + "platform": "cuda", + "image_tag": "maxusai/ollama:sync-0.34.4-main", + "started_utc": "20260927T213816", + "expectations_file": "expectations.toml", + "schema_version": 1, + "version": "0.34.4-dynres-0-gb43ee8e", + "profile": "cuda-dynres-903", + "patchset": [ + "001", + "002", + "004", + "005", + "903", + "908" + ], + "container": "ollama-0.34.4-dynres-0-gb43ee8e", + "llama_cpp_build": "161755f29", + "mlx_build": "0.32.2-65-g59d600b" + }, + "results": [ + { + "check": "version", + "arch": null, + "status": "PASS", + "summary": "0.34.4-dynres-0-gb43ee8e matches profile cuda-dynres-903", + "expected": "^0\\.3[234]\\.\\d+(-rc\\d+)?-dynres(\\.\\d+)?(-\\d+-g[0-9a-f]{7,40}|-[0-9a-f]{7,40})$", + "actual": "0.34.4-dynres-0-gb43ee8e" + }, + { + "check": "payload_pin", + "arch": null, + "status": "PASS", + "summary": "llama.cpp build 161755f29 matches the measured payload (docker exec)", + "expected": "161755f29", + "actual": "161755f29" + }, + { + "check": "toolchain_pin", + "arch": null, + "status": "SKIP", + "summary": "profile records no toolchain_build to assert against", + "diagnosis": "Add toolchain_build (e.g. \"rocm-7.2.4\" or \"cuda-13.0\") so a ROCm/CUDA bump under an unchanged payload fails loudly instead of inheriting these expectations. Platforms with no such toolchain (metal, cpu) leave it unset and keep this skip; the MLX library is pinned separately by mlx_build." + }, + { + "check": "image_tag", + "arch": null, + "status": "PASS", + "summary": "ollama-0.34.4-dynres-0-gb43ee8e runs maxusai/ollama:sync-0.34.4-main", + "expected": "maxusai/ollama:sync-0.34.4-main", + "actual": "maxusai/ollama:sync-0.34.4-main" + }, + { + "check": "go_patch_marker", + "arch": null, + "status": "PASS", + "summary": "--image-max-tokens present in binary (1)", + "expected": 1, + "actual": 1 + }, + { + "check": "metal_tensor_host", + "arch": null, + "status": "SKIP", + "summary": "does not apply on cuda" + }, + { + "check": "metal_tensor_runtime", + "arch": null, + "status": "SKIP", + "summary": "does not apply on cuda" + }, + { + "check": "poison_probe", + "arch": null, + "status": "PASS", + "summary": "1.06x-ceiling trigger decodes healthily (52 chars, done_reason='stop'); slot clean after; no node-level check (meter off \u2014 set OLLAMA_CLIP_NODE_STATS=ffn_down to enable)", + "actual": "The image is a black and white checkerboard pattern." + }, + { + "check": "text_baseline", + "arch": "nemotron_h_omni", + "status": "PASS", + "summary": "in-image text prefix = 19 (calibrated on 256x144+512x288); text-only = 20; text-only is +1 and MUST NOT be used (B8)", + "actual": 19 + }, + { + "check": "token_ladder", + "arch": "nemotron_h_omni", + "status": "PASS", + "summary": "5/5 geometries within +/-2", + "expected": { + "256x144": 266, + "512x288": 266, + "1024x576": 578, + "2048x1152": 2306, + "3072x1728": 3270 + }, + "actual": { + "256x144": 266, + "512x288": 266, + "1024x576": 578, + "2048x1152": 2306, + "3072x1728": 3270 + }, + "rows": [ + { + "size": "256x144", + "expected": 266, + "actual": 266, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + }, + { + "size": "512x288", + "expected": 266, + "actual": 266, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + }, + { + "size": "1024x576", + "expected": 578, + "actual": 578, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + }, + { + "size": "2048x1152", + "expected": 2306, + "actual": 2306, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + }, + { + "size": "3072x1728", + "expected": 3270, + "actual": 3270, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + } + ], + "shape": "scaling", + "expected_shape": "dynamic" + }, + { + "check": "payload_proof", + "arch": "nemotron_h_omni", + "status": "PASS", + "summary": "budget logged as custom on max/min (min 256*32^2=262144, max 3328*32^2=3407872)", + "expected": "min 256*32^2=262144, max 3328*32^2=3407872", + "actual": { + "min": "262144 (custom value)", + "max": "3407872 (custom value)" + } + }, + { + "check": "aspect_ladder", + "arch": "nemotron_h_omni", + "status": "SKIP", + "summary": "no aspect-ladder expectation recorded" + }, + { + "check": "think_format", + "arch": "nemotron_h_omni", + "status": "PASS", + "summary": "valid JSON after thinking (486 tokens)", + "expected": "non-empty JSON response with think:true + format:json", + "actual": { + "response_chars": 256, + "thinking_chars": 1623, + "eval_count": 486 + } + }, + { + "check": "pinned_image_token_budget", + "arch": "nemotron_h_omni", + "status": "PASS", + "summary": "pinned 3328 -> 3270 (ceiling 3328, +2 markers)", + "expected": 3270, + "actual": 3270, + "arms": [ + { + "arm": "pinned", + "pin": 3328, + "expected": 3270, + "actual": 3270, + "ok": true + }, + { + "arm": "ceiling_invariant", + "expected": "grid <= 3328", + "actual": 3268, + "ok": true + }, + { + "arm": "unpinned_control", + "expected": 2306, + "actual": 2306, + "ok": true + } + ] + }, + { + "check": "text_baseline", + "arch": "gemma4", + "status": "PASS", + "summary": "in-image text prefix = 19 (calibrated on 256x144+512x288); text-only = 19", + "actual": 19 + }, + { + "check": "token_ladder", + "arch": "gemma4", + "status": "PASS", + "summary": "5/5 geometries within +/-2", + "expected": { + "256x144": 1102, + "512x288": 1102, + "1024x576": 1102, + "2048x1152": 1102, + "3072x1728": 1102 + }, + "actual": { + "256x144": 1102, + "512x288": 1102, + "1024x576": 1102, + "2048x1152": 1102, + "3072x1728": 1102 + }, + "rows": [ + { + "size": "256x144", + "expected": 1102, + "actual": 1102, + "delta": 0, + "ok": true, + "queue_wait_s": 0.0 + }, + { + "size": "512x288", + "expected": 1102, + "actual": 1102, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + }, + { + "size": "1024x576", + "expected": 1102, + "actual": 1102, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + }, + { + "size": "2048x1152", + "expected": 1102, + "actual": 1102, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + }, + { + "size": "3072x1728", + "expected": 1102, + "actual": 1102, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + } + ], + "shape": "flat", + "expected_shape": "flat" + }, + { + "check": "payload_proof", + "arch": "gemma4", + "status": "PASS", + "summary": "budget logged as custom on max/min (min 70*48^2=161280, max 1120*48^2=2580480)", + "expected": "min 70*48^2=161280, max 1120*48^2=2580480", + "actual": { + "min": "161280 (custom value)", + "max": "2580480 (custom value)" + } + }, + { + "check": "aspect_ladder", + "arch": "gemma4", + "status": "SKIP", + "summary": "no aspect-ladder expectation recorded" + }, + { + "check": "think_format", + "arch": "gemma4", + "status": "PASS", + "summary": "valid JSON after thinking (266 tokens)", + "expected": "non-empty JSON response with think:true + format:json", + "actual": { + "response_chars": 216, + "thinking_chars": 867, + "eval_count": 266 + } + }, + { + "check": "pinned_image_token_budget", + "arch": "gemma4", + "status": "PASS", + "summary": "pinned 560 -> 529 (ceiling 1120, +2 markers)", + "expected": 529, + "actual": 529, + "arms": [ + { + "arm": "pinned", + "pin": 560, + "expected": 529, + "actual": 529, + "ok": true + }, + { + "arm": "ceiling_invariant", + "expected": "grid <= 1120", + "actual": 527, + "ok": true + } + ] + }, + { + "check": "text_baseline", + "arch": "qwen35", + "status": "PASS", + "summary": "in-image text prefix = 13 (calibrated on 256x144+512x288); text-only = 13", + "actual": 13 + }, + { + "check": "token_ladder", + "arch": "qwen35", + "status": "PASS", + "summary": "5/5 geometries within +/-2", + "expected": { + "256x144": 1034, + "512x288": 1034, + "1024x576": 1034, + "2048x1152": 2306, + "3072x1728": 4082 + }, + "actual": { + "256x144": 1034, + "512x288": 1034, + "1024x576": 1034, + "2048x1152": 2306, + "3072x1728": 4082 + }, + "rows": [ + { + "size": "256x144", + "expected": 1034, + "actual": 1034, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + }, + { + "size": "512x288", + "expected": 1034, + "actual": 1034, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + }, + { + "size": "1024x576", + "expected": 1034, + "actual": 1034, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + }, + { + "size": "2048x1152", + "expected": 2306, + "actual": 2306, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + }, + { + "size": "3072x1728", + "expected": 4082, + "actual": 4082, + "delta": 0, + "ok": true, + "queue_wait_s": 0.01 + } + ], + "shape": "scaling", + "expected_shape": "dynamic" + }, + { + "check": "payload_proof", + "arch": "qwen35", + "status": "PASS", + "summary": "budget logged as custom on min (min 1024*32^2=1048576, max 4096*32^2=4194304)", + "expected": "min 1024*32^2=1048576, max 4096*32^2=4194304", + "actual": { + "min": "1048576 (custom value)", + "max": "4194304" + } + }, + { + "check": "aspect_ladder", + "arch": "qwen35", + "status": "SKIP", + "summary": "no aspect-ladder expectation recorded" + }, + { + "check": "think_format", + "arch": "qwen35", + "status": "PASS", + "summary": "valid JSON after thinking (324 tokens)", + "expected": "non-empty JSON response with think:true + format:json", + "actual": { + "response_chars": 414, + "thinking_chars": 923, + "eval_count": 324 + } + }, + { + "check": "pinned_image_token_budget", + "arch": "qwen35", + "status": "SKIP", + "summary": "pinned budget does not apply to this arch: image_min_tokens/image_max_tokens are arch-gated to gemma4 and nemotron_h_omni; qwen35's visionServerArgs branch ignores them." + }, + { + "check": "metal_tensor_payload", + "arch": null, + "status": "SKIP", + "summary": "does not apply on cuda" + }, + { + "check": "mlx_payload_pin", + "arch": null, + "status": "PASS", + "summary": "MLX build 59d600b matches the measured payload (0.32.2-65-g59d600b), from the shipped libmlx.so \u2014 no MLX request ran in this window, so binary/payload skew is not covered", + "expected": "59d600b5e64c238427d0f8d897ab7c682ef4d3d2", + "actual": "0.32.2-65-g59d600b" + }, + { + "check": "endpoint_exclusive", + "arch": null, + "status": "PASS", + "summary": "no contention detected (worst queue wait 0.0s)", + "expected": "queue wait <= 10.0s", + "actual": "0.0s", + "models_loaded": [ + "qwen3.8:27b-q4_K_M", + "gemma4:31b-it-q4_K_M" + ] + } + ], + "summary": { + "PASS": 21, + "SKIP": 8 + } +} \ No newline at end of file