No linux/arm64 images are available

[jriberg]

The documentation lists that linux/arm64 images are available, but running this command:
"""
docker run --rm --platform=linux/arm64 -p 1235:8080 \
-v "$HOME/.lmstudio/models/lmstudio-community/Qwen3-VL-4B-Instruct-GGUF:/models"
ghcr.io/ggml-org/llama.cpp:light
-m /models/Qwen3-VL-4B-Instruct-Q4_K_M.gguf
--host 0.0.0.0 --port 8080
"""

Then I get:
"""
Unable to find image 'ghcr.io/ggml-org/llama.cpp:light' locally
light: Pulling from ggml-org/llama.cpp
Digest: sha256:dec7889fcc3e60418f3ec20828649b2fd947f0d3bc657f17350741dd9e0174c8
Status: Image is up to date for ghcr.io/ggml-org/llama.cpp:light
docker: Error response from daemon: image with reference ghcr.io/ggml-org/llama.cpp:light was found but does not provide the specified platform (linux/arm64)

Run 'docker run --help' for more information
"""

So, linux/arm64 is not acctually built?

(Sorry, codeblocks didn't work for some reason)

[taronaeo]

If I'm not wrong, linux/arm64 platforms were built previously, but disabled for some reason.

Let me check the CI.

Edit: Here

llama.cpp/.github/workflows/docker.yml

Lines 38 to 40 in f3dd7b8

	# Multi-stage build
	# Note: the arm64 images are failing, which prevents the amd64 images from being built
	# https://github.com/ggml-org/llama.cpp/issues/11888

I don't think anyone went back to check if its working now.

[angt]

If it can help, you can find optimized builds for Linux/ARM on installama.sh :)

curl https://installama.sh | sh

[mags0ft]

If I'm not wrong, linux/arm64 platforms were built previously, but disabled for some reason.

Whoops, those got disabled a bit too silently - things started to fail on my end for no apparent reason until I found this thread. Any idea when they will work again?

[taronaeo]

I don't think we have anyone looking into this. If you're interested, you can play around with .github/workflows/docker.yml and test it on your GitHub CI to see if its working now. If it is, feel free to submit a PR :)

[Ts-sound]

After investigation and testing, I identified two issues causing the build failure:

1. Incorrect ARM Architecture Detection in CMake: The build system incorrectly detects and uses armv9.2-a architecture features
2. Outdated QEMU Version: The QEMU version has a known bug that causes random segmentation faults during ARM64 compilation

Error Logs

Issue 1: Invalid ARM Architecture Flags

...
#13 17.89 -- ARM detected
#13 17.90 -- Checking for ARM features using flags:
#13 17.90 --   -march=armv9.2-a+dotprod+fp16+sve+i8mm+sve2+sme
#13 17.90 -- Adding CPU backend variant ggml-cpu-armv9.2_2: -march=armv9.2-a+dotprod+fp16+sve+i8mm+sve2+sme GGML_USE_DOTPROD;GGML_USE_FP16_VECTOR_ARITHMETIC;GGML_USE_SVE;GGML_USE_MATMUL_INT8;GGML_USE_SVE2;GGML_USE_SME
#13 17.94 -- ggml version: 0.9.8
...
cc1: error: unknown value 'armv9.2-a+dotprod+fp16+sve+i8mm+sve2+sme' for '-march'
#13 144.2 cc1: note: valid arguments are: armv8-a armv8.1-a armv8.2-a armv8.3-a armv8.4-a armv8.5-a armv8.6-a armv8-r native
#13 144.2 gmake[2]: *** [ggml/src/CMakeFiles/ggml-cpu-armv9.2_2.dir/build.make:76: ggml/src/CMakeFiles/ggml-cpu-armv9.2_2.dir/ggml-cpu/ggml-cpu.c.o] Error 1
#13 144.3 gmake[1]: *** [CMakeFiles/Makefile2:1358: ggml/src/CMakeFiles/ggml-cpu-armv9.2_2.dir/all] Error 2
#13 144.3 gmake[1]: *** Waiting for unfinished jobs....

Issue 2: QEMU Segmentation Fault

#13 873.3 [ 41%] Building CXX object src/CMakeFiles/llama.dir/models/jais2.cpp.o
#13 877.6 [ 42%] Building CXX object src/CMakeFiles/llama.dir/models/jamba.cpp.o
#13 882.3 [ 42%] Building CXX object src/CMakeFiles/llama.dir/models/kimi-linear.cpp.o
#13 882.7 c++: internal compiler error: Segmentation fault signal terminated program cc1plus
#13 882.7 Please submit a full bug report,
#13 882.7 with preprocessed source if appropriate.
#13 882.7 See <file:///usr/share/doc/gcc-11/README.Bugs> for instructions.
#13 882.7 gmake[2]: *** [src/CMakeFiles/llama.dir/build.make:1168: src/CMakeFiles/llama.dir/models/kimi-linear.cpp.o] Error 4
#13 882.7 gmake[2]: *** Waiting for unfinished jobs....
#13 888.3 gmake[1]: *** [CMakeFiles/Makefile2:608: src/CMakeFiles/llama.dir/all] Error 2
#13 888.4 gmake: *** [Makefile:136: all] Error 2
#13 ERROR: process "/bin/sh -c if [ \"$TARGETARCH\" = \"amd64\" ] || [ \"$TARGETARCH\" = \"arm64\" ]; then         cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=OFF -DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_EXAMPLES=OFF -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF;     else         echo \"Unsupported architecture\";         exit 1;     fi &&     cmake --build build -j $(nproc)" did not complete successfully: exit code: 2

Root Cause Analysis

1. Incorrect ARM Architecture Detection

The CMake build system detects ARM architecture but incorrectly assumes armv9.2-a features are available. The compiler only supports up to armv8.6-a:

valid arguments are: armv8-a armv8.1-a armv8.2-a armv8.3-a armv8.4-a armv8.5-a armv8.6-a armv8-r native

2. QEMU Bug

The outdated QEMU version has a known bug that causes random segmentation faults during ARM64 compilation. This issue has been observed in other projects as well.

Solution

changes: 1ada337

Fix 1: Update QEMU Version

Upgrade to the latest QEMU version:

tonistiigi/binfmt:qemu-v10.2.1

Fix 2: Disable armv9.2 Detection in CMake

Temporarily disable the ARM v9.2 architecture detection in CMake configuration.

Verification

After applying both fixes, the build completes successfully.

Successful Build Run: https://github.com/Ts-sound/llama.cpp/actions/runs/23428791649

[Ts-sound]

@taronaeo

[taronaeo]

Feel free to create a PR featuring these changes, including upgrading the compiler version :)

[ehfd]

I affirm that the below (using GCC 14) works well for cuda-new.Dockerfile. I believe that aarch64 builds for CUDA can be enabled as well, and it will help a lot.

build-and-push-job:
  stage: build-and-push
  variables:
    LLAMA_CPP_REF: "b8477"
  script:
  - cd $CI_PROJECT_DIR
  - git clone https://github.com/ggml-org/llama.cpp.git
  - cd llama.cpp
  - git checkout $LLAMA_CPP_REF
  - |
  - sed -i 's/apt-get install -y build-essential/apt-get install -y gcc-14 g++-14 build-essential/g' .devops/cuda-new.Dockerfile
  - sed -i '/apt-get install -y gcc-14 g++-14 build-essential/a ENV CC=gcc-14 CXX=g++-14 CUDAHOSTCXX=g++-14' .devops/cuda-new.Dockerfile
  - |
    # Build cuda-new.Dockerfile
    docker buildx build \
      -f .devops/cuda-new.Dockerfile \
      --push \
      --provenance=false \
      --platform linux/arm64 \
      --target full \
      -t $CI_REGISTRY_IMAGE:latest \
      -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA \
      .