Skip to content

ci(bump-hrx): merge AMD's pin into our line instead of rebasing 159 commits (fixes the 5-day silent sync failure) - #328

Merged
bong-water-water-bong merged 2 commits into
mainfrom
1bit/bump-hrx-merge-sync
Oct 6, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
1bit/bump-hrx-merge-sync

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

What was wrong

bump-hrx failed on every scheduled run from 2026-10-01 to 2026-10-05, always at the "Sync the fork and rebase our commits" step, with "Open the bump PR" skipped. Step 2 "Require the token" passes, so this was never auth.

Measured cause. Our 1bit/hrx-vulkan-patched carries 159 commits that are genuinely ours — git cherry against AMD's pin reports 0 already upstream — and AMD's pin had moved 70 commits over the same files. Rebasing replays all 159 and died at commit 1 of 159:

Rebasing (1/159) Auto-merging …/dispatch-mul-mat.cpp
CONFLICT (content): Merge conflict in …/dispatch-mul-mat.cpp
error: could not apply d7b4d78f7... ggml-hrx: IQ3_XXS matvec kernel for MUL_MAT

The same two lines merge with only 15 conflicting files, all under ggml/src/ggml-hrx/ — so the premise ("rebase our commits onto AMD's pin") is what's wrong, not the plumbing.

What this changes

  • Merge instead of rebase. git merge --no-edit --no-ff rather than rebase --onto, so one bounded resolution replaces a 159-step replay.
  • Failures now speak. A failing run files (or comments on) an issue naming the conflicting files. This workflow broke silently for five days and the HRX pin froze; that is the actual defect, and it should not be able to recur.
  • Unchanged safety: the ancestry guard on 1bit/hrx-vulkan, the tag created before the branch moves (so the commit the engine pins stays reachable), and the --force-with-lease push.

Remaining step, stated plainly

This makes the automation correct, not the divergence resolved. The next run will merge and still hit those 15 files once, because AMD has changed our own HRX files (including flash_attention_decode_split_f32_f16_wmma.loom, the kernel from #314). Resolving them is a one-off manual job that needs HRX validation on Strix Halo — it is not on the release-critical path, because the release needs only the narrow #315 pin bump, which does not move to AMD's pin at all.

1bit release bot added 2 commits October 6, 2026 04:35
…ommits

bump-hrx failed on every scheduled run from 2026-10-01 to 2026-10-05 at the
"Sync the fork and rebase our commits" step, with "Open the bump PR" skipped.
The token step passes, so it was never auth.

Measured cause: our 1bit/hrx-vulkan-patched carries 159 commits that are
genuinely ours (`git cherry` against AMD's pin reports 0 already upstream), and
AMD's pin had moved 70 commits over the same files. Rebasing replays all 159 and
died at commit 1 of 159:

  Rebase (1/159) Auto-merging .../dispatch-mul-mat.cpp
  CONFLICT (content): ... could not apply d7b4d78f7 (IQ3_XXS matvec kernel)

Merging the same two lines conflicts in 15 files, all under ggml/src/ggml-hrx/,
and resolves once. So the premise ("rebase our commits onto AMD's pin") is what
was wrong, not the plumbing.

Also: a failing scheduled job that tells no one is the real defect here — the
HRX pin silently froze for five days. A failing run now files or updates an
issue naming the conflicting files.

The rebase path's ancestry guard, the tag-before-move safety, and the
force-with-lease push are unchanged, so the commit the engine pins stays
reachable exactly as before.
AMD's pins come as a PAIR, but this workflow moved only llama.cpp, and
.gitmodules records hrx-system as "rebased by hand". So every AMD move left
hrx-system silently behind — the same class of silent drift that let the
llama.cpp pin freeze for five days.

Adds a step that merges AMD's hrx-system pin into 1bit-MONSTER/hrx-system
1bit/main with the same merge-not-rebase treatment, tags the previous tip
before moving the branch, and force-with-lease pushes. On conflict it names the
conflicting files and fails.

Also reads the branch into an explicit ref rather than FETCH_HEAD: after two
fetches a bare FETCH_HEAD silently compares the wrong pair — that bug cost a
debugging cycle on the llama.cpp half.

The bump PR now moves BOTH gitlinks instead of naming hrx-system as a manual
follow-up.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant