Skip to content

ci: fix the ctest sanitize runs - #26593

Merged
ggerganov merged 6 commits into
ggml-org:masterfrom
netrunnereve:ci
Aug 10, 2026
Merged

ggerganov merged 6 commits into
ggml-org:masterfrom
netrunnereve:ci

Conversation

@netrunnereve

Copy link
Copy Markdown
Contributor

Those jobs are getting stuck as we only have three machines with the cpu tag.

Requirements

@netrunnereve
netrunnereve requested a review from a team as a code owner August 4, 2026 17:16
@github-actions github-actions Bot added the devops improvements to build systems and github actions label Aug 4, 2026
@netrunnereve
netrunnereve marked this pull request as draft August 4, 2026 17:26
@netrunnereve

Copy link
Copy Markdown
Contributor Author

Aaaand it crashed, back to draft. Actually the thread test seems to have crashed sometimes in the past as well.

@netrunnereve
netrunnereve marked this pull request as ready for review August 4, 2026 19:54
@netrunnereve
netrunnereve marked this pull request as draft August 4, 2026 19:55
@netrunnereve

Copy link
Copy Markdown
Contributor Author

From looking at the old jobs it looks like the thread tests only pass on ggml-6-x86-vulkan-t4 and ggml-8-x86-nvidia-a10 while it crashes on sg-hl1-ci-nvidia-vulkan-cm. It crashes on my own computer as well with the same ThreadSanitizer: unexpected memory mapping error and it's probably due to how ASLR is setup on that particular machine.

So right now I've made that particular test only run on those two specific machines and we can always add more to the list or have a special tag for it in the future. Or maybe this test can be straight up removed altogether, I dunno.

@netrunnereve
netrunnereve marked this pull request as ready for review August 5, 2026 17:17
@netrunnereve

Copy link
Copy Markdown
Contributor Author

Okay I think it makes more sense to just run the thread test on the Github machines instead of hardcoding a possibly changing list of self hosted machines to run it on.

@netrunnereve netrunnereve changed the title ci: run sanitize on more machines ci: fix the ctest sanitize runs Aug 5, 2026
Comment thread .github/workflows/build-sanitize.yml
@netrunnereve netrunnereve added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Aug 8, 2026
@ggerganov
ggerganov merged commit aea252f into ggml-org:master Aug 10, 2026
6 checks passed
@CISC

CISC commented Aug 10, 2026

Copy link
Copy Markdown
Member

@netrunnereve
netrunnereve deleted the ci branch August 10, 2026 15:55
@netrunnereve

Copy link
Copy Markdown
Contributor Author

Strange, this started failing for no reason: https://github.com/ggml-org/llama.cpp/actions/runs/31362342045/job/93373543781#step:7:30

Ugh I think it's something to do with the Intel-LNL-U7-258V machine, I checked the jobs after my pr got merged and all the ones that failed were on that machine. I'll deal with this later.

huaxel pushed a commit to huaxel/CachyLLama that referenced this pull request Aug 12, 2026
* Update build-sanitize.yml

* make it run on pr

* fix thread

* Update build-sanitize.yml

* Update build-sanitize.yml

* just run thread on github machine
brittlewis12 pushed a commit to brittlewis12/llama.cpp that referenced this pull request Aug 17, 2026
* Update build-sanitize.yml

* make it run on pr

* fix thread

* Update build-sanitize.yml

* Update build-sanitize.yml

* just run thread on github machine
thecodacus pushed a commit to thecodacus/llama.cpp that referenced this pull request Sep 7, 2026
* Update build-sanitize.yml

* make it run on pr

* fix thread

* Update build-sanitize.yml

* Update build-sanitize.yml

* just run thread on github machine
zbrad pushed a commit to zbrad/llama.cpp that referenced this pull request Sep 10, 2026
* Update build-sanitize.yml

* make it run on pr

* fix thread

* Update build-sanitize.yml

* Update build-sanitize.yml

* just run thread on github machine
pl752 pushed a commit to pl752/llama.cpp that referenced this pull request Sep 15, 2026
* Update build-sanitize.yml

* make it run on pr

* fix thread

* Update build-sanitize.yml

* Update build-sanitize.yml

* just run thread on github machine
frostyautumnleaf pushed a commit to frostyautumnleaf/llama.cpp that referenced this pull request Oct 5, 2026
* Update build-sanitize.yml

* make it run on pr

* fix thread

* Update build-sanitize.yml

* Update build-sanitize.yml

* just run thread on github machine
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops improvements to build systems and github actions merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants