Skip to content

docs(fold): gates 4–6 on gfx1151 for v0.35.0 — production's GPU code, and rocm7 admits the fold - #434

Merged
glennneuber merged 3 commits into
task/upstream-sync-0.35.0from
docs/fold-0350-gfx1151
Oct 1, 2026
Merged

glennneuber merged 3 commits into
task/upstream-sync-0.35.0from
docs/fold-0350-gfx1151

Conversation

@glennneuber

Copy link
Copy Markdown

This PR adds gfx1151's record of gates 4–6 to #427's task doc, and widens the rocm7 preflight profile so that it admits the fold. It targets #427's branch, not main. Everything in it was measured on gfx1151 only. In the CUDA host's sections it changes only what concerns gfx1151: gfx1151's entries in the status table's rows 4–6, and the rocm7 row of "Preflight profiles". Rows 4 and 5 now also say which host their existing text is about.

What

The new section is "Gates 4–6 on gfx1151 (2026-09-30)" in docs/maxusai/tasks/upstream-sync-0.35.0.md. The fold ran in production's environment (f16 KV cache, flash attention on, the two-pass flow, two slots), and was compared with production's own build, 0.34.4-dynres-0-gb43ee8e.

gate gfx1151
4, image A full build of fedbe0507, maxusai-ollama:0.34.4-dynres-41-gfedbe05-rocm7-gfx1151. One native library of five differs, libggml-hip.so, and its .hip_fatbin (all of its gfx1151 GPU code) is byte-identical to production's. 805 adds 64 bytes of host code, a switch that nobody sets
5, preflight PASS=27 SKIP=5. 30 of 32 rows equal production's run (#415), measured values included. The other two name the build
the new surface System One answers on qwen3.6 and refuses qwen3.8. A typical_p request gets a 200 and a deprecation warning, and the value reaches llama-server's sampler. The promoted image refuses the same request with a 400
6, think off 0 of 4,902 scored cells differ, over five models. All 140 answers are byte-identical
6, OCRBench 172 = 172. All 200 answers and their token counts are identical
6, think on No model loses a finish. qwen3.6 and gemma4:26b are byte-identical, answers and thinking, including gemma4:26b's three loops to 131072, which are the baseline's three. gemma4:31b: one quality move, toward the fold. qwen3.8: every rate identical. nemotron3: rates within the draws, all Fisher p ≥ 0.57

Files:

  • docs/maxusai/tasks/upstream-sync-0.35.0.md: the section and the table entries above. Every output in the section is pasted verbatim from the tool that made it.
  • docs/maxusai/vision-suite/preflight/expectations.toml: rocm7-0-34-4-dynres's version_pattern admits 0.35.0 as well as 0.34.[34], as ADR 0032 says for a fold that moves no native input. 0.35.1 stays refused, because its rc0 moves llama.cpp. The patchset lists 805, because every build since docs: rocBLAS on gfx1151 — both runtime routes closed, and why #371 carries it.
  • docs/maxusai/vision-suite/preflight/test_verdicts.py: TestRocm7ProfileAdmitsTheV0350Fold pins both changes. The old pattern fails it.
  • docs/maxusai/vision-suite/preflight/runs/preflight-rocm7-0350-fold-gfedbe05.json: the gate 5 run record. It is force-added past the runs/ ignore, as the other cited records are.

Finding: in the suite, two-pass think-on does not repeat byte for byte

gemma4:31b runs greedy, and both arms send the same prompts, sampler and llama-server flags. Yet its thinking differs in 17 of 28 cells.

  • Cold captures of three of those cells give the same pass-1 thinking on both images, byte for byte.
  • Pass 2 goes to either slot, by timing. llama-server gives it pass 1's slot ("by LCP similarity") if pass 1 has released that slot, and otherwise the other, empty slot ("by LRU"). Production's own image did both in three cold captures, so the race is not the fold's.
  • So on gfx1151, gemma4:31b's think-on is compared by finishes, rates and quality moves, not by bytes, on either image. The race shows on gemma4:31b only: qwen3.6 and gemma4:26b repeat byte for byte.

A correction

My gate 6a comment on #427 quoted a cmp_raw.py that compared two keys the answer files do not have, so it would report "byte-identical" for any two runs. The corrected tool compares file bytes. It gives the same result, 140 of 140, and it finds the differences on gemma4:31b's think-on. The correction is on #427 (comment).

Verification

  • test_verdicts.py: 200 tests OK. tomllib loads expectations.toml.
  • check_source_paths.py --changed-since origin/task/upstream-sync-0.35.0: every referenced file resolves.
  • check_no_names.py with a local deny-list file, over the tree and this branch's commits: no match.

amd-server/rocm-gfx1151

🤖 Generated with Claude Code

glennneuber and others added 3 commits September 30, 2026 20:57
The v0.35.0 fold (#427) moves no native input: llama.cpp stays b11081. As ADR 0032 says, its builds share the
rocm7 profile, and the version pattern widens from 0.34.[34] to admit 0.35.0 as well. 0.35.1 stays refused,
because its rc0 moves llama.cpp to b11232.

805 entered main with #371, after the promoted build, and the ROCm build applies it. So the patch set lists it,
as it came to list 908. On gfx1151 it changes no GPU code. The fold image's libggml-hip.so has a byte-identical
.hip_fatbin (60,336,816 bytes) and 64 more bytes of host .text, the GGML_CUDA_FATTN_FORCE_TILE switch.

TestRocm7ProfileAdmitsTheV0350Fold pins the widening:
- It admits production's stamp, the fold's interim stamp and the fold's tag stamp.
- It refuses 0.35.1, a previous payload, a dirty tree and a point tag.
- The resolver maps the tag stamp to this profile.
- The patch set lists 805.

The old pattern fails the admit test. test_verdicts.py passes 200 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fold image, maxusai-ollama:0.34.4-dynres-41-gfedbe05-rocm7-gfx1151, was built from fedbe05. The run used
production's environment on a bench container: f16, flash attention on, two-pass and two slots. It ran preflight
with --quality. The interim stamp resolves to rocm7-0-34-4-dynres. The result is PASS=27 SKIP=5, VERDICT PASS.

Against the promoted image's last run (preflight-rocm7-0344-aspect-gb43ee8e.json), 30 of 32 rows are identical,
every measured value included. The two other rows are version and image_tag, which name the image. The record's
patchset line is the profile's, which did not list 805 at run time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fold ran on gfx1151 in production's environment (f16 KV cache, flash attention on, the two-pass flow, two
slots) and was compared with production's own build, 0.34.4-dynres-0-gb43ee8e.

- Gate 4: a full build of fedbe05. One native library of five differs, libggml-hip.so. Its .hip_fatbin, all of
  its gfx1151 GPU code, is byte-identical to production's. 805 adds 64 bytes of host code.
- Gate 5: PASS=27 SKIP=5. 30 of 32 rows equal production's run (#415), measured values included.
- System One answers on qwen3.6 and refuses qwen3.8. A typical_p request gets a 200, and the value reaches
  llama-server's sampler.
- Gate 6, think off: 0 of 4,902 scored cells differ. All 140 answers and all 200 OCRBench answers are
  byte-identical.
- Gate 6, think on: no model loses a finish. qwen3.6 and gemma4:26b are byte-identical, their loops included.
  gemma4:31b differs in the suite because of the two-pass flow's slot race. Cold, its pass-1 thinking is
  byte-identical on both images, and production's own image puts pass 2 on either slot.

The status table gets gfx1151's entries in rows 4-6, and the profiles table the widened rocm7 row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@glennneuber
glennneuber merged commit 218b526 into task/upstream-sync-0.35.0 Oct 1, 2026
3 checks passed
@glennneuber

Copy link
Copy Markdown
Author

Review of #434 at 15326a5b4, by the CUDA host at the maintainer's request: OK. Merged into the fold branch (218b526f1).

Checked:

  • The run record.
  • The profile.
    • The widened version_pattern admits 0.34.3, 0.34.4 and 0.35.0 stamps. It refuses 0.35.1, an rc, 0.35.10 and 0.34.2.
    • TestRocm7ProfileAdmitsTheV0350Fold passes on this branch. Against the fold head's expectations.toml it fails (2 failures, 1 error), so it pins both changes.
    • test_verdicts.py passes on the full tree.
  • The slot race.
    • Cold, pass-1 thinking is byte-identical in all three cells.
    • In captures 1 and 2, pass 2 took the same slot on both images, and the answers match.
    • In capture 3, scene_single_anchored, the two images used different slots, and the answers differ by 2 tokens. That is the cell of gemma4:31b's one quality move, and the doc already attributes the move to the run's state, not the fold.
  • The CUDA sections are unchanged.
    • The CUDA text of rows 4–6 is as it was.
    • So are the cuda and mlx-cuda profile rows.
    • Only gfx1151's entries are added.

CUDA production cannot hit this race. The race needs two slots, and all 7 of its llama-server launches since the 0.34.4 deploy ran -np 1.

ai-server/mlx-cuda

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant