[SYCL] Fix get mem error by arthw · Pull Request #28227 · ggml-org/llama.cpp

arthw · 2026-09-02T06:14:03Z

Overview

Fixed the issue #28134 involved in #27968.

Additional information

The PR [SYCL] Enhance to get the free memory of Intel GPU #27968 use level zero API to get free memory size to support more GPUs. But it can't work on WSL on windows, because level zero Sysman doesn't support it.
The solution in this PR fallback to SYCL API when it's not supported.
Add GGML_SYCL_DEV_DEBUG() for development debug.

Verified on Linux (Arc770,UHD Graphics 770 in 13700K), Windows (140T) and WSL (Ubuntu) on Windows.

Requirements

I have read and agree with the contributing guidelines
AI usage disclosure: NO

Co-authored-by: Titaniumtown <titaniumtown@proton.me>

strtgbb · 2026-09-02T15:18:39Z

This does not compile for me. The base b10743 does compile.

Compiler log

- The C compiler identification is IntelLLVM 2026.0.0
-- The CXX compiler identification is IntelLLVM 2026.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/intel/oneapi/compiler/2026.0/bin/icx - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/intel/oneapi/compiler/2026.0/bin/icpx - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- llama.cpp version: 0.3.0-dev
CMAKE_BUILD_TYPE=Release
-- Found Git: /usr/bin/git (found version "2.55.0")
-- The ASM compiler identification is IntelLLVM
-- Found assembler: /opt/intel/oneapi/compiler/2026.0/bin/icx
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- ccache found, compilation results will be cached. Disable with GGML_CCACHE=OFF.
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- GGML_SYSTEM_ARCH: x86
-- Found OpenMP_C: -fiopenmp (found version "5.1")
-- Found OpenMP_CXX: -fiopenmp (found version "5.1")
-- Found OpenMP: TRUE (found version "5.1")
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native 
-- Using RPC backend
--   RDMA transport enabled (auto-detected)
-- Including RPC backend
-- GGML_SYCL_TARGET=INTEL
-- Performing Test SUPPORTS_SYCL
-- Performing Test SUPPORTS_SYCL - Success
-- Using oneAPI Release SYCL compiler (icpx).
-- SYCL found
-- SYCL Compiler version: 20260000
-- SYCL_INCLUDE_DIR: /opt/intel/oneapi/compiler/2026.0/include
-- SYCL_LIBRARY=/opt/intel/oneapi/compiler/2026.0/lib/libsycl.so
-- Found IntelSYCL: /opt/intel/oneapi/compiler/2026.0/include (found version "202012")
-- GGML_SYCL_SUPPORT_LEVEL_ZERO_API ON
CMake Warning at ggml/src/ggml-sycl/CMakeLists.txt:121 (message):
  Level Zero loader or headers not found, Level Zero support disabled


-- Found oneDNN: /opt/intel/oneapi/dnnl/2026.0/lib/libdnnl.so.3.11
-- Skipping -ze-intel-greater-than-4GB-buffer-required for spir64_gen AOT
-- MKL_VERSION: 2026.0.0
-- MKL_ROOT: /opt/intel/oneapi/mkl/2026.0
-- MKL_ARCH: intel64
-- MKL_SYCL_LINK: None, set to ` dynamic` by default
-- MKL_LINK: None, set to ` dynamic` by default
-- MKL_SYCL_INTERFACE_FULL: None, set to ` intel_ilp64` by default
-- MKL_INTERFACE_FULL: None, set to ` intel_ilp64` by default
-- MKL_SYCL_THREADING: None, set to ` tbb_thread` by default
-- MKL_THREADING: None, set to ` intel_thread` by default
-- MKL_SYCL_MPI: None, set to ` intelmpi` by default
-- MKL_MPI: None, set to ` intelmpi` by default
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_scalapack_ilp64.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_cdft_core.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_intel_ilp64.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_intel_thread.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_core.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_blacs_intelmpi_ilp64.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_distributed_dft.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_blas.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_lapack.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_dft.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_sparse.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_data_fitting.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_rng.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_stats.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_vm.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_tbb_thread.so
-- Found /opt/intel/oneapi/tbb/2023.0/lib/intel64/gcc4.8/libtbb.so.12
-- Found /opt/intel/oneapi/compiler/2026.0/lib/libiomp5.so
-- find GGML_SYCL_GRAPH
-- find GGML_SYCL_HOST_MEM_FALLBACK
-- GGML_SYCL_DEVICE_ARCH=bmg_g21 (AOT via spir64_gen)
-- Including SYCL backend
-- ggml version: 0.22.0
-- ggml commit:  d2f23593f
-- Found OpenSSL: /usr/lib/libcrypto.so (found version "3.6.4")
-- Performing Test OPENSSL_VERSION_SUPPORTED
-- Performing Test OPENSSL_VERSION_SUPPORTED - Success
-- OpenSSL found: 3.6.4
-- Generating embedded license file for target: llama-app
-- Configuring done (2.8s)
-- Generating done (0.3s)
-- Build files have been written to: ~/llama.cpp/build.sycl.b10743-5-gd2f23593f
12
[  0%] Building CXX object vendor/cpp-httplib/CMakeFiles/cpp-httplib.dir/httplib.cpp.o
[  0%] Building CXX object common/CMakeFiles/llama-common-base.dir/build-info.cpp.o
[  0%] Building CXX object tools/mtmd/CMakeFiles/llama-llava-cli.dir/deprecation-warning.cpp.o
[  0%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[  0%] Building CXX object vendor/hash/CMakeFiles/vendor-hash.dir/hash.cpp.o
[  1%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml.cpp.o
[  2%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[  2%] Building CXX object tools/ui/CMakeFiles/llama-ui-embed.dir/embed.cpp.o
[  2%] Building CXX object tools/mtmd/CMakeFiles/llama-qwen2vl-cli.dir/deprecation-warning.cpp.o
[  2%] Building CXX object tools/mtmd/CMakeFiles/llama-gemma3-cli.dir/deprecation-warning.cpp.o
[  2%] Building C object vendor/hash/CMakeFiles/vendor-hash.dir/xxhash/xxhash.c.o
[  2%] Building CXX object tools/mtmd/CMakeFiles/llama-minicpmv-cli.dir/deprecation-warning.cpp.o
[  2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[  2%] Linking CXX executable llama-ui-embed
[  2%] Linking CXX static library libllama-common-base.a
[  2%] Building CXX object vendor/hash/CMakeFiles/vendor-hash.dir/sha1/sha1.c.o
[  3%] Linking CXX executable ../../bin/llama-minicpmv-cli
[  3%] Linking CXX executable ../../bin/llama-gemma3-cli
[  3%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend-meta.cpp.o
[  4%] Linking CXX executable ../../bin/llama-qwen2vl-cli
[  4%] Linking CXX executable ../../bin/llama-llava-cli
[  4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[  4%] Building C object vendor/hash/CMakeFiles/vendor-hash.dir/sha256/sha256.c.o
[  4%] Linking CXX static library libcpp-httplib.a
[  4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[  5%] Linking CXX static library libvendor-hash.a
[  5%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[  6%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[  6%] Built target llama-common-base
[  6%] Built target cpp-httplib
[  6%] Linking CXX shared library ../../bin/libggml-base.so
[  6%] Built target vendor-hash
[  6%] Built target llama-gemma3-cli
[  6%] Built target llama-llava-cli
[  6%] Built target llama-qwen2vl-cli
[  6%] Built target llama-minicpmv-cli
[  6%] Built target llama-ui-embed
[  6%] Provisioning UI assets
-- UI: downloading from b10748: https://huggingface.co/buckets/ggml-org/llama-ui/resolve/b10748/dist.tar.gz
[  6%] Built target ggml-base
[  6%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/quants.c.o
[  6%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o
[  6%] Building CXX object ggml/src/ggml-rpc/CMakeFiles/ggml-rpc.dir/transport.cpp.o
[  6%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o
[  7%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/repack.cpp.o
[  7%] Building CXX object ggml/src/ggml-rpc/CMakeFiles/ggml-rpc.dir/ggml-rpc.cpp.o
[  7%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/iqp.cpp.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/hbm.cpp.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/traits.cpp.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o
[  8%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/ggml-sycl.cpp.o
[  8%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/add-id.cpp.o
[  8%] Linking CXX shared library ../../../bin/libggml-rpc.so
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/binary-ops.cpp.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/unary-ops.cpp.o
[  9%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/binbcast.cpp.o
[  9%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/col2im-1d.cpp.o
[  9%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/common.cpp.o
[  9%] Built target ggml-rpc
[ 10%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/vec.cpp.o
-- UI: download dist.tar.gz from b10748 failed: "HTTP response code said error"
-- UI: downloading from latest: https://huggingface.co/buckets/ggml-org/llama-ui/resolve/latest/dist.tar.gz
[ 10%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/concat.cpp.o
[ 10%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/conv.cpp.o
[ 10%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/conv2d-dw.cpp.o
[ 10%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ops.cpp.o
[ 10%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/llamafile/sgemm.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/conv2d-transpose.cpp.o
[ 11%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/arch/x86/quants.c.o
[ 11%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/arch/x86/repack.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/conv2d.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/conv3d.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/convert.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/count-equal.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/cpy.cpp.o
-- UI: archive verified and extracted
-- UI: HF download succeeded, stamp updated (latest)
-- UI: gzip compression applied (~/llama.cpp/build.sycl.b10743-5-gd2f23593f/tools/ui/dist/_gzip)
embed: write output file ~/llama.cpp/build.sycl.b10743-5-gd2f23593f/tools/ui/ui.h
embed: write output file ~/llama.cpp/build.sycl.b10743-5-gd2f23593f/tools/ui/ui.cpp
[ 11%] Built target llama-ui-assets
[ 11%] Building CXX object tools/ui/CMakeFiles/llama-ui.dir/ui.cpp.o
[ 11%] Linking CXX static library libllama-ui.a
[ 11%] Built target llama-ui
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/cross_entropy_loss.cpp.o
[ 12%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/cumsum.cpp.o
[ 12%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/diag.cpp.o
[ 12%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/dmmv.cpp.o
[ 12%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/dsv4-hc.cpp.o
[ 12%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/element_wise.cpp.o
[ 12%] Linking CXX shared library ../../bin/libggml-cpu.so
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:314:8: error: unknown type name 'ze_result_t'; did you mean 'ur_result_t'?
  314 | static ze_result_t init_zes() {
      |        ^~~~~~~~~~~
      |        ur_result_t
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:956:3: note: 'ur_result_t' declared here
  956 | } ur_result_t;
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:315:5: error: unknown type name 'ze_result_t'; did you mean 'ur_result_t'?
  315 |     ze_result_t res = zesInit(0);
      |     ^~~~~~~~~~~
      |     ur_result_t
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:956:3: note: 'ur_result_t' declared here
  956 | } ur_result_t;
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:315:23: error: use of undeclared identifier 'zesInit'
  315 |     ze_result_t res = zesInit(0);
      |                       ^~~~~~~
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:316:16: error: use of undeclared identifier 'ZE_RESULT_SUCCESS'; did you mean 'UR_RESULT_SUCCESS'?
  316 |     if (res != ZE_RESULT_SUCCESS) {
      |                ^~~~~~~~~~~~~~~~~
      |                UR_RESULT_SUCCESS
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:787:3: note: 'UR_RESULT_SUCCESS' declared here
  787 |   UR_RESULT_SUCCESS = 0,
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:323:1: error: unknown type name 'ze_result_t'; did you mean 'ur_result_t'?
  323 | ze_result_t get_zes_init_res() {
      | ^~~~~~~~~~~
      | ur_result_t
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:956:3: note: 'ur_result_t' declared here
  956 | } ur_result_t;
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:324:12: error: unknown type name 'ze_result_t'; did you mean 'ur_result_t'?
  324 |     static ze_result_t zes_init_res = init_zes();
      |            ^~~~~~~~~~~
      |            ur_result_t
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:956:3: note: 'ur_result_t' declared here
  956 | } ur_result_t;
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:727:32: warning: unused parameter 'device_dst' [-Wunused-parameter]
  727 | static void dev2dev_memcpy(int device_dst, sycl::queue &q_dst, int device_src, sycl::queue &q_src, void *ptr_dst,
      |                                ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:727:68: warning: unused parameter 'device_src' [-Wunused-parameter]
  727 | static void dev2dev_memcpy(int device_dst, sycl::queue &q_dst, int device_src, sycl::queue &q_src, void *ptr_dst,
      |                                                                    ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:5651:35: error: use of undeclared identifier 'ZE_RESULT_SUCCESS'; did you mean 'UR_RESULT_SUCCESS'?
 5651 |             get_zes_init_res() == ZE_RESULT_SUCCESS ? (MemoryAPIType) g_ggml_sycl_get_mem_api : MemoryAPIType::MEMORY_API_TYPE_SYCL);
      |                                   ^~~~~~~~~~~~~~~~~
      |                                   UR_RESULT_SUCCESS
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:787:3: note: 'UR_RESULT_SUCCESS' declared here
  787 |   UR_RESULT_SUCCESS = 0,
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:6078:31: error: use of undeclared identifier 'ZE_RESULT_SUCCESS'; did you mean 'UR_RESULT_SUCCESS'?
 6078 |         get_zes_init_res() == ZE_RESULT_SUCCESS ? (MemoryAPIType) g_ggml_sycl_get_mem_api : MemoryAPIType::MEMORY_API_TYPE_SYCL);
      |                               ^~~~~~~~~~~~~~~~~
      |                               UR_RESULT_SUCCESS
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:787:3: note: 'UR_RESULT_SUCCESS' declared here
  787 |   UR_RESULT_SUCCESS = 0,
      |   ^
2 warnings and 8 errors generated.
make[2]: *** [ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/build.make:79: ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/ggml-sycl.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 12%] Built target ggml-cpu
make[1]: *** [CMakeFiles/Makefile2:2690: ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

arthw · 2026-09-02T15:26:38Z

@strtgbb

-- GGML_SYCL_SUPPORT_LEVEL_ZERO_API ON
CMake Warning at ggml/src/ggml-sycl/CMakeLists.txt:121 (message):
  Level Zero loader or headers not found, Level Zero support disabled

The root cause is you didn't install the level-zero dev package.
Because the GGML_SYCL_SUPPORT_LEVEL_ZERO_API ON is default.
It trigger the build error.

Two methods:
1.
Install level-zero-dev by sudo apt-get install -y libze-dev, refer to https://dgpu-docs.intel.com/installation-guides/installing-packages-from-the-intel-ppa.html.

Disable using level-zero by -DGGML_SYCL_SUPPORT_LEVEL_ZERO_API=OFF in build command.

Thank you!

strtgbb · 2026-09-02T16:42:33Z

Thanks, I ran sudo pacman -S level-zero-headers and the build works now.

manderss99 · 2026-09-05T07:50:19Z

Correction (edited): this comment originally said my machine was the same hardware as #28134 (Core Ultra 9 288V / Arc 140V). That was wrong — mine is a Core Ultra 7 255H (Arrow Lake-H) with Arc 140T, i915, Ubuntu 26.04, oneAPI 2026.1. So the results below are a different GPU from the one in #28134, and they do not tell you whether #28134 itself is resolved on Lunar Lake. Apologies for the noise.

I hit the same abort as #28134 on an Arc 140T, so I tested this PR merged onto master 4d9176092. Two things to report: a build break in the PR, and what actually triggered the abort on my machine.

1. The PR does not compile when `GGML_SYCL_SUPPORT_LEVEL_ZERO_API` is undefined

init_zes() / get_zes_init_res() in ggml-sycl.cpp, and the ternary at both get_memory_size() call sites, use ze_result_t / zesInit / ZE_RESULT_SUCCESS outside any #ifdef. mem.cpp guards the same symbols, so only ggml-sycl.cpp breaks — 8 errors with icpx 2026.1:

ggml/src/ggml-sycl/ggml-sycl.cpp:317:8:   error: unknown type name 'ze_result_t'; did you mean 'ur_result_t'?
ggml/src/ggml-sycl/ggml-sycl.cpp:318:23:  error: use of undeclared identifier 'zesInit'
ggml/src/ggml-sycl/ggml-sycl.cpp:319:16:  error: use of undeclared identifier 'ZE_RESULT_SUCCESS'
ggml/src/ggml-sycl/ggml-sycl.cpp:326:1:   error: unknown type name 'ze_result_t'
ggml/src/ggml-sycl/ggml-sycl.cpp:5672:35: error: use of undeclared identifier 'ZE_RESULT_SUCCESS'
ggml/src/ggml-sycl/ggml-sycl.cpp:6112:31: error: use of undeclared identifier 'ZE_RESULT_SUCCESS'

This is easy to hit by accident. GGML_SYCL_SUPPORT_LEVEL_ZERO_API defaults to ON, but ggml/src/ggml-sycl/CMakeLists.txt only adds the compile definition when it finds both level_zero/ze_api.h and an unversioned libze_loader.so. On Ubuntu those come from libze-dev, which the Intel GPU runtime packages do not pull in. With only libze1 installed, configure prints

CMake Warning: Level Zero loader or headers not found, Level Zero support disabled

while GGML_SYCL_SUPPORT_LEVEL_ZERO_API:BOOL=ON still sits in CMakeCache.txt — the cache variable looks enabled but the define is absent.

Guarding the helpers and routing both call sites through one accessor fixes it:

#ifdef GGML_SYCL_SUPPORT_LEVEL_ZERO_API
static ze_result_t init_zes() { /* ... */ }

ze_result_t get_zes_init_res() {
    static ze_result_t zes_init_res = init_zes();
    return zes_init_res;
}

static MemoryAPIType ggml_sycl_effective_mem_api() {
    return get_zes_init_res() == ZE_RESULT_SUCCESS ? (MemoryAPIType) g_ggml_sycl_get_mem_api :
                                                     MemoryAPIType::MEMORY_API_TYPE_SYCL;
}
#else
static MemoryAPIType ggml_sycl_effective_mem_api() {
    // no Level Zero headers/loader at build time: sysman is unavailable by construction
    return MemoryAPIType::MEMORY_API_TYPE_SYCL;
}
#endif

then get_memory_size(..., ggml_sycl_effective_mem_api()); at both sites. With that, the PR builds and the abort is gone.

2. On my machine the abort was caused by building without Level Zero support

My build had no Level Zero headers, so GGML_SYCL_SUPPORT_LEVEL_ZERO_API was undefined and the zesInit(0) in initialize_sycl_begining() was compiled out. Sysman is then never initialised in the process, sycl::ext::intel::info::device::free_memory throws, get_memory_size_by_sycl_api() returns false, and master aborts. This also explains why ZES_ENABLE_SYSMAN=1 did not help — on this driver the env var alone is not enough, the process has to call zesInit().

After apt install libze-dev and re-running CMake, unpatched master 4d9176092 runs correctly on the Arc 140T:

$ ./build-master/bin/llama-cli --list-devices
  SYCL0: Intel(R) Arc(TM) Graphics (58670 MiB, 27532 MiB free)

$ ./build-master/bin/llama-bench -m Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf --device SYCL0 -ngl 99 -fa on
| qwen35moe 35B.A3B Q5_K - Medium |  25.28 GiB |  35.51 B | SYCL | 99 | 1 | SYCL0 |  pp16 | 22.32 |
| qwen35moe 35B.A3B Q5_K - Medium |  25.28 GiB |  35.51 B | SYCL | 99 | 1 | SYCL0 |   tg8 | 12.01 |

Both memory APIs now return a real free figure rather than falling back to total — GGML_SYCL_GET_MEM_API=0 and =1 agree (58670 MiB total / 27532 MiB free), so sysman works on this iGPU once zesInit() has run. (@arthw — you listed 140T on Windows as verified; this is the Linux/i915 side of the same GPU.)

One CMake gotcha for anyone reproducing this: find_path/find_library cache their NOTFOUND results, so installing the package is not enough — the cache entries have to be cleared, e.g. cmake -B build -U LEVEL_ZERO_INCLUDE_DIR -U ZE_LOADER_LIB.

So this PR is still valuable — it removes the hard abort and makes every path end in a usable value — but the practical fix for at least this configuration is having the Level Zero loader present at build time. Since the PR already touches docs/backend/SYCL.md, it might be worth mentioning there that the Level Zero development package (libze-dev on Ubuntu) is needed for GGML_SYCL_SUPPORT_LEVEL_ZERO_API to actually take effect, and perhaps promoting the "Level Zero support disabled" message so it is harder to miss.

Disclosure: I used Claude Code for the investigation and to write the guard patch; all build and run results above were produced and verified on my machine.

arthw · 2026-09-05T15:35:12Z

@manderss99
Thank you for your test and sharing!
This feature is designed to with/without level zero.
I need to update it to adapt to without level zero case.
In Intel linux driver guide, level-zero-dev is recommended to be installed.
But user still could skip to install it.

I will update the PR as soon.

Thank you!

…detect the error

arthw · 2026-09-07T03:31:44Z

@manderss99
It's fixed now.
Could you help test it again?

Thank you!

manderss99 · 2026-09-07T04:17:55Z

Retested at a53449ab0 on an Arc 140T (Core Ultra 7 255H, Arrow Lake-H, i915, Ubuntu). The build error is fixed and there is no abort in either configuration. Thanks for the quick turnaround.

Build

configuration	result
Level Zero dev package present (`GGML_SYCL_SUPPORT_LEVEL_ZERO_API` defined)	builds clean
no Level Zero dev package (macro undefined)	builds clean — this was 8 compile errors before

With the development headers missing, configure now prints the new message and continues instead of failing:

CMake Warning at ggml/src/ggml-sycl/CMakeLists.txt:
  Level Zero loader or development headers not found, Level Zero API support
  disabled.  Please install the Level Zero SDK/development package to support
  Level Zero API features. ...

(Reproduced by forcing LEVEL_ZERO_DEV_INCLUDE_DIR empty. The no-Level-Zero binaries below were built with -DGGML_SYCL_SUPPORT_LEVEL_ZERO_API=OFF, which gives the identical compile condition.)

Runtime — all four combinations fine, no `GGML_ABORT`

With Level Zero:

GGML_SYCL_GET_MEM_API=0   SYCL0: Intel(R) Arc(TM) Graphics (62575 MiB, 19744 MiB free)
GGML_SYCL_GET_MEM_API=1   SYCL0: Intel(R) Arc(TM) Graphics (62553 MiB, 19712 MiB free)

| qwen35moe 35B.A3B Q5_K - Medium | 25.28 GiB | SYCL | ngl 99 | fa 1 | pp16 | 21.23 |
| qwen35moe 35B.A3B Q5_K - Medium | 25.28 GiB | SYCL | ngl 99 | fa 1 |  tg8 | 12.31 |

Without Level Zero:

GGML_SYCL_GET_MEM_API=0   SYCL0: Intel(R) Arc(TM) Graphics (87494 MiB, 22966 MiB free)
GGML_SYCL_GET_MEM_API=1   SYCL0: Intel(R) Arc(TM) Graphics (87494 MiB, 28597 MiB free)

| qwen35moe 35B.A3B Q5_K - Medium | 25.28 GiB | SYCL | ngl 99 | fa 1 | pp16 | 21.47 |
| qwen35moe 35B.A3B Q5_K - Medium | 25.28 GiB | SYCL | ngl 99 | fa 1 |  tg8 | 11.79 |

Both paths return a real free figure (free != total), so the SYCL fallback is doing its job, and performance is the same within this machine's noise.

Moving zesInit() into a function-local static behind get_zes_init_res() also removes the ordering hazard I ran into: the query now works even when ggml_backend_sycl_device_get_memory() runs before initialize_sycl_begining().

Two small things

1. The zesInit failure warning can never print. init_zes() logs through GGML_SYCL_DEV_DEBUG(), but g_ggml_sycl_dev_debug is assigned in ggml_check_sycl() after initialize_sycl_begining(), and the static inside get_zes_init_res() is initialised on exactly that first call. The flag is therefore still 0 when the message would be emitted, and setting GGML_SYCL_DEV_DEBUG=1 does not bring it back. Master printed this unconditionally on std::cerr. On WSL — the configuration this PR targets — a failing zesInit() is now completely silent. Parsing the environment variables before initialize_sycl_begining(), or keeping this one warning unconditional, would fix it.

2. (pre-existing, not introduced here) GGML_SYCL_GET_MEM_API has no effect on the --list-devices path. In the numbers above, =0 and =1 both report ~62.5 GiB total in the Level Zero build, while the Level-Zero-less build reports 87.5 GiB total from the SYCL API — if the variable were taking effect, =1 should have matched that 87.5 GiB figure. No GGML_SYCL_DEBUG output appears on that path either. ggml_backend_sycl_device_get_memory() is reached during device enumeration before ggml_check_sycl() has parsed the variables, so it always sees the default. Same on master. Worth a look, since the docs point users at this variable as the workaround when the memory query misbehaves.

Disclosure: I used Claude Code for the builds and tests and to draft this comment; all numbers above are from runs on my machine.

arthw · 2026-09-07T10:28:35Z

@manderss99
Thank you for your feedback!

They are fixed:

remove GGML_SYCL_DEV_DEBUG() in all codes.
call env vars in all entrys, support GGML_SYCL_GET_MEM_API in --list-devices.

Thank you!

ahmetem · 2026-09-11T20:45:23Z

when will it merge ?

arthw · 2026-09-12T13:42:12Z

@ahmetem
It's marked as "merge ready".
Wait for owner to final review and merge.

If there are more feedback to verify this PR, the merge will be speeded up.

Thank you!

arthw · 2026-09-12T15:36:40Z

@Titaniumtown @ggerganov
Could you review it and merge as soon?

Thank you!

* fix for unsupport zes API * optimize the code * adjust the log level * rm unused head files * Update docs/backend/SYCL.md Co-authored-by: Titaniumtown <titaniumtown@proton.me> * fix the error to detect level zero SDK/dev package, stop build after detect the error * update the message * fix the build error when missed to install level zero dev package * rm GGML_SYCL_DEV_DEBUG, mv read env vars in all entry functions --------- Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com> Co-authored-by: Titaniumtown <titaniumtown@proton.me> Co-authored-by: Neo Zhang <NA>

arthw and others added 4 commits September 2, 2026 00:11

fix for unsupport zes API

1d3f075

optimize the code

db36812

adjust the log level

381e7b5

rm unused head files

cbdb6d5

arthw requested a review from a team as a code owner September 2, 2026 06:14

arthw mentioned this pull request Sep 2, 2026

Eval bug: SYCL backend aborts at model load on Lunar Lake iGPU (Arc 140V) — device memory query fails with both Level Zero and SYCL APIs #28134

Closed

github-actions Bot added documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels Sep 2, 2026

Titaniumtown reviewed Sep 2, 2026

View reviewed changes

Comment thread docs/backend/SYCL.md Outdated

Update docs/backend/SYCL.md

d2f2359

Co-authored-by: Titaniumtown <titaniumtown@proton.me>

arthw mentioned this pull request Sep 2, 2026

[SYCL]Sysman free-memory query may be unavailable #28239

Open

arthw mentioned this pull request Sep 4, 2026

Misc. bug: SYCL backend fails GPU offload on Intel Arc Graphics (Meteor Lake 125H) with UR_RESULT_ERROR_INVALID_VALUE at ggml-sycl.cpp:1800 #28314

Open

arthw and others added 3 commits September 6, 2026 23:23

fix the error to detect level zero SDK/dev package, stop build after …

9a0e49a

…detect the error

update the message

8a53e0a

fix the build error when missed to install level zero dev package

a53449a

Neo Zhang and others added 2 commits September 7, 2026 18:23

rm GGML_SYCL_DEV_DEBUG, mv read env vars in all entry functions

19b054c

Merge branch 'master' into fix_get_mem

76bcad6

This was referenced Sep 7, 2026

Eval bug: SYCL backend core dumps with A770 - "failed to get device memory size" #28519

Closed

Compile bug: ggml_backend_sycl_device_get_memory aborts with "failed to get device memory size" on Intel Arc Pro B60 (Battlemage) #28515

Open

arthw added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Sep 10, 2026

arthw mentioned this pull request Sep 12, 2026

Eval bug: SYCL: bad output on Qwen3.6 35B A3B #28728

Open

ggerganov merged commit b6b003d into ggml-org:master Sep 13, 2026
28 of 30 checks passed

jintakhan mentioned this pull request Sep 14, 2026

Eval bug: SYCL demands extreme scratchpad allocation (2GB+) when ngram-mod is enabled #28860

Open

ggerganov mentioned this pull request Sep 14, 2026

ggml : bump version to 0.24.0 ggml-org/ggml#1627

Merged

BrewTestBot mentioned this pull request Sep 14, 2026

ggml 0.24.0 Homebrew/homebrew-core#310890

Merged

1 task done

SomeSunlight mentioned this pull request Sep 19, 2026

Stabilize Intel SYCL / Level Zero inference on WSL2 SomeSunlight/ai-workstation#8

Closed

ServeurpersoCom mentioned this pull request Sep 23, 2026

Eval bug: Nemotron-3-Nano-30B-A3B fails to load only in server mode. #29259

Closed

Navigation Menu

Repository navigation

[SYCL] Fix get mem error - #28227

[SYCL] Fix get mem error#28227
ggerganov merged 10 commits into
ggml-org:masterfrom
arthw:fix_get_mem

arthw commented Sep 2, 2026

Uh oh!

Uh oh!

strtgbb commented Sep 2, 2026

Uh oh!

arthw commented Sep 2, 2026

Uh oh!

strtgbb commented Sep 2, 2026

Uh oh!

manderss99 commented Sep 5, 2026 •

edited

Loading

Uh oh!

arthw commented Sep 5, 2026

Uh oh!

arthw commented Sep 7, 2026

Uh oh!

manderss99 commented Sep 7, 2026

Uh oh!

arthw commented Sep 7, 2026

Uh oh!

ahmetem commented Sep 11, 2026

Uh oh!

arthw commented Sep 12, 2026

Uh oh!

arthw commented Sep 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

arthw commented Sep 2, 2026

Overview

Additional information

Requirements

Uh oh!

Uh oh!

strtgbb commented Sep 2, 2026

Uh oh!

arthw commented Sep 2, 2026

Uh oh!

strtgbb commented Sep 2, 2026

Uh oh!

manderss99 commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. The PR does not compile when GGML_SYCL_SUPPORT_LEVEL_ZERO_API is undefined

2. On my machine the abort was caused by building without Level Zero support

Uh oh!

arthw commented Sep 5, 2026

Uh oh!

arthw commented Sep 7, 2026

Uh oh!

manderss99 commented Sep 7, 2026

Build

Runtime — all four combinations fine, no GGML_ABORT

Two small things

Uh oh!

arthw commented Sep 7, 2026

Uh oh!

ahmetem commented Sep 11, 2026

Uh oh!

arthw commented Sep 12, 2026

Uh oh!

arthw commented Sep 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

manderss99 commented Sep 5, 2026 •

edited

Loading

1. The PR does not compile when `GGML_SYCL_SUPPORT_LEVEL_ZERO_API` is undefined

Runtime — all four combinations fine, no `GGML_ABORT`