Repository navigation
Eval bug: GPT-OSS-120B: Vulkan backend fails to allocate KV cache with OOM error, despite enough free memory #15120
Description
Name and Version
[docker@dd353b48e141 ~]$ llama-server --version
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Radeon 8060S Graphics (AMD proprietary driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
version: 6096 (fd1234c)
built with cc (GCC) 15.1.1 20250729 for x86_64-pc-linux-gnu
Operating systems
Linux
GGML backends
Vulkan
Hardware
AMD 395+ Strix Halo APU with 8060s iGPU
Models
GPT-OSS-120B
Problem description & steps to reproduce
llama-server -m ~/.cache/llama.cpp/ggml-org_gpt-oss-120b-GGUF_gpt-oss-120b-mxfp4-00001-of-00003.gguf -c 0 --host 0.0.0.0 --port 9000 -ngl 999
This gives the following error:
ggml_vulkan: Device memory allocation of size 17482395648 failed.
ggml_vulkan: Requested buffer size exceeds device memory allocation limit: ErrorOutOfDeviceMemory
ggml_gallocr_reserve_n: failed to allocate Vulkan0 buffer of size 17482395648
The model itself is under 70GB, and there is 128GB of GTT memory available. This issue seems specific to GPT-OSS-120B, since bigger models (Such as Qwen3-235B in Q3_K_XL) do not show the same issue.
First Bad Commit
No response
Relevant log output
ggml_vulkan: Device memory allocation of size 17482395648 failed.
ggml_vulkan: Requested buffer size exceeds device memory allocation limit: ErrorOutOfDeviceMemory
ggml_gallocr_reserve_n: failed to allocate Vulkan0 buffer of size 17482395648
Activity
Mushoz commentedon Aug 6, 2025
Important to note is that this happens with amdvlk, radv and the proprietary pro drivers, so it's not driver specific either.
LokeshSN commentedon Aug 6, 2025
Facing the same issue. Logs:
.\llama.cpp\llama-server.exe -m .\Models\GGUFs\ggml-org\gpt-oss-120b\gpt-oss-120b-mxfp4-00001-of-00003.gguf -ngl 99 --threads -1 -c 20000 --jinja --reasoning-format none -ot '([1-5]+).ffn_.*_exps.=CPU'
load_backend: loaded RPC backend from E:\EverythingAI\llama.cpp\ggml-rpc.dll
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = Radeon Instinct MI60 (AMD proprietary driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: none
ggml_vulkan: 1 = Radeon Instinct MI60 (AMD proprietary driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: none
load_backend: loaded Vulkan backend from E:\EverythingAI\llama.cpp\ggml-vulkan.dll
load_backend: loaded CPU backend from E:\EverythingAI\llama.cpp\ggml-cpu-skylakex.dll
build: 6101 (0d88315) with clang version 19.1.5 for x86_64-pc-windows-msvc
system info: n_threads = 40, n_threads_batch = 40, total_threads = 40
system_info: n_threads = 40 (n_threads_batch = 40) / 80 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
main: binding port with default address family
main: HTTP server is listening, hostname: 127.0.0.1, port: 8080, http threads: 39
main: loading model
srv load_model: loading model '.\Models\GGUFs\ggml-org\gpt-oss-120b\gpt-oss-120b-mxfp4-00001-of-00003.gguf'
llama_model_load_from_file_impl: using device Vulkan0 (Radeon Instinct MI60) - 32496 MiB free
llama_model_load_from_file_impl: using device Vulkan1 (Radeon Instinct MI60) - 32496 MiB free
llama_model_loader: additional 2 GGUFs metadata loaded.
llama_model_loader: loaded meta data with 35 key-value pairs and 687 tensors from .\Models\GGUFs\ggml-org\gpt-oss-120b\gpt-oss-120b-mxfp4-00001-of-00003.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = gpt-oss
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Prerelease 100 120b Hf
llama_model_loader: - kv 3: general.finetune str = hf
llama_model_loader: - kv 4: general.basename str = prerelease-100
llama_model_loader: - kv 5: general.size_label str = 120B
llama_model_loader: - kv 6: gpt-oss.block_count u32 = 36
llama_model_loader: - kv 7: gpt-oss.context_length u32 = 131072
llama_model_loader: - kv 8: gpt-oss.embedding_length u32 = 2880
llama_model_loader: - kv 9: gpt-oss.feed_forward_length u32 = 2880
llama_model_loader: - kv 10: gpt-oss.attention.head_count u32 = 64
llama_model_loader: - kv 11: gpt-oss.attention.head_count_kv u32 = 8
llama_model_loader: - kv 12: gpt-oss.rope.freq_base f32 = 150000.000000
llama_model_loader: - kv 13: gpt-oss.attention.layer_norm_rms_epsilon f32 = 0.000010
llama_model_loader: - kv 14: gpt-oss.expert_count u32 = 128
llama_model_loader: - kv 15: gpt-oss.expert_used_count u32 = 4
llama_model_loader: - kv 16: gpt-oss.attention.key_length u32 = 64
llama_model_loader: - kv 17: gpt-oss.attention.value_length u32 = 64
llama_model_loader: - kv 18: gpt-oss.attention.sliding_window u32 = 128
llama_model_loader: - kv 19: gpt-oss.expert_feed_forward_length u32 = 2880
llama_model_loader: - kv 20: gpt-oss.rope.scaling.type str = yarn
llama_model_loader: - kv 21: gpt-oss.rope.scaling.factor f32 = 32.000000
llama_model_loader: - kv 22: gpt-oss.rope.scaling.original_context_length u32 = 4096
llama_model_loader: - kv 23: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 24: tokenizer.ggml.pre str = gpt-4o
llama_model_loader: - kv 25: tokenizer.ggml.tokens arr[str,201088] = ["!", """, "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 26: tokenizer.ggml.token_type arr[i32,201088] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 27: tokenizer.ggml.merges arr[str,446189] = ["─á ─á", "─á ─á─á─á", "─á─á ─á─á", "...
llama_model_loader: - kv 28: tokenizer.ggml.eos_token_id u32 = 199999
llama_model_loader: - kv 29: tokenizer.chat_template str = {#-\n In addition to the normal input...
llama_model_loader: - kv 30: general.quantization_version u32 = 2
llama_model_loader: - kv 31: general.file_type u32 = 38
llama_model_loader: - kv 32: split.no u16 = 0
llama_model_loader: - kv 33: split.tensors.count i32 = 687
llama_model_loader: - kv 34: split.count u16 = 3
llama_model_loader: - type f32: 433 tensors
llama_model_loader: - type q8_0: 146 tensors
llama_model_loader: - type mxfp4: 108 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = MXFP4 MoE
print_info: file size = 59.02 GiB (4.34 BPW)
load: printing all EOG tokens:
load: - 199999 ('<|endoftext|>')
load: - 200002 ('<|return|>')
load: - 200007 ('<|end|>')
load: - 200012 ('<|call|>')
load: special_eog_ids contains both '<|return|>' and '<|call|>' tokens, removing '<|end|>' token from EOG list
load: special tokens cache size = 21
load: token to piece cache size = 1.3332 MB
print_info: arch = gpt-oss
print_info: vocab_only = 0
print_info: n_ctx_train = 131072
print_info: n_embd = 2880
print_info: n_layer = 36
print_info: n_head = 64
print_info: n_head_kv = 8
print_info: n_rot = 64
print_info: n_swa = 128
print_info: is_swa_any = 1
print_info: n_embd_head_k = 64
print_info: n_embd_head_v = 64
print_info: n_gqa = 8
print_info: n_embd_k_gqa = 512
print_info: n_embd_v_gqa = 512
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-05
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 0.0e+00
print_info: n_ff = 2880
print_info: n_expert = 128
print_info: n_expert_used = 4
print_info: causal attn = 1
print_info: pooling type = 0
print_info: rope type = 2
print_info: rope scaling = yarn
print_info: freq_base_train = 150000.0
print_info: freq_scale_train = 0.03125
print_info: n_ctx_orig_yarn = 4096
print_info: rope_finetuned = unknown
print_info: model type = ?B
print_info: model params = 116.83 B
print_info: general.name = Prerelease 100 120b Hf
print_info: n_ff_exp = 2880
print_info: vocab type = BPE
print_info: n_vocab = 201088
print_info: n_merges = 446189
print_info: BOS token = 11 ','
print_info: EOS token = 199999 '<|endoftext|>'
print_info: EOT token = 199999 '<|endoftext|>'
print_info: LF token = 198 '─è'
print_info: EOG token = 199999 '<|endoftext|>'
print_info: EOG token = 200002 '<|return|>'
print_info: EOG token = 200012 '<|call|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: offloading 36 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 37/37 layers to GPU
load_tensors: Vulkan0 model buffer size = 15099.74 MiB
load_tensors: Vulkan1 model buffer size = 12394.09 MiB
load_tensors: CPU_Mapped model buffer size = 26926.73 MiB
load_tensors: CPU_Mapped model buffer size = 24666.72 MiB
....................................................................................................
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 20000
llama_context: n_ctx_per_seq = 20000
llama_context: n_batch = 2048
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = 0
llama_context: kv_unified = false
llama_context: freq_base = 150000.0
llama_context: freq_scale = 0.03125
llama_context: n_ctx_per_seq (20000) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
llama_context: Vulkan_Host output buffer size = 0.77 MiB
llama_kv_cache_unified_iswa: creating non-SWA KV cache, size = 20000 cells
llama_kv_cache_unified: Vulkan0 KV buffer size = 351.56 MiB
llama_kv_cache_unified: Vulkan1 KV buffer size = 351.56 MiB
llama_kv_cache_unified: size = 703.12 MiB ( 20000 cells, 18 layers, 1/1 seqs), K (f16): 351.56 MiB, V (f16): 351.56 MiB
llama_kv_cache_unified_iswa: creating SWA KV cache, size = 640 cells
llama_kv_cache_unified: Vulkan0 KV buffer size = 12.50 MiB
llama_kv_cache_unified: Vulkan1 KV buffer size = 10.00 MiB
llama_kv_cache_unified: size = 22.50 MiB ( 640 cells, 18 layers, 1/1 seqs), K (f16): 11.25 MiB, V (f16): 11.25 MiB
ggml_vulkan: Device memory allocation of size 2696488960 failed.
ggml_vulkan: Requested buffer size exceeds device memory allocation limit: ErrorOutOfDeviceMemory
ggml_gallocr_reserve_n: failed to allocate Vulkan0 buffer of size 2696488960
graph_reserve: failed to allocate compute buffers
llama_init_from_model: failed to initialize the context: failed to allocate compute pp buffers
common_init_from_params: failed to create context with model '.\Models\GGUFs\ggml-org\gpt-oss-120b\gpt-oss-120b-mxfp4-00001-of-00003.gguf'
srv load_model: failed to load model, '.\Models\GGUFs\ggml-org\gpt-oss-120b\gpt-oss-120b-mxfp4-00001-of-00003.gguf'
srv operator(): operator(): cleaning up before exit...
main: exiting due to model loading error
0cc4m commentedon Aug 6, 2025
@Mushoz Please post the whole log in a gist.
@LokeshSN That's different, Mushoz had an attempted 17GB allocation, which is sadly infeasible for Vulkan. We have to check whether it is necessary for the model to run or some other issue. You have a 2.7GB allocation, which would not be particularly unusual. The only reason that is failing for you is the 2GB allocation limit of the AMD proprietary driver. With radv it would be working, since that has a 4GB allocation limit, similar to most other Vulkan drivers.
Mushoz commentedon Aug 6, 2025
@0cc4m here is the full log with the
--verboseflag included: https://gist.github.com/Mushoz/2a4a29e091d2fc7f320ac659510730b40cc4m commentedon Aug 6, 2025
That does look like the kv-cache is just far beyond the max allocation your driver supports. Can you try reducing the ctx size? Also, did you disable SWA?
Mushoz commentedon Aug 6, 2025
At half (64k) context it crashes with a OOM error for an attempted allocation of over 8 GB. At a quarter (32k) it crashes with an OOM error for an attempted allocation of just over 4GB. Bringing that down to 28k resulted in the following allocation which succeeded:
I didn't do such a thing, at least not that I know off? Should I disable that? If so, how?
XZiar commentedon Aug 6, 2025
try add
-fato enable flash attention? It could significantly reducecompute buffer sizewhen using long context (at least on CUDA).jeffbolznv commentedon Aug 6, 2025
If you're going to try fa you should grab #15126
0cc4m commentedon Aug 7, 2025
SWA is not off, it just wasn't shown because the regular KV buffer failed to allocate before the SWA buffer would have been allocated.
illioren commentedon Aug 7, 2025
I have the same issue (also running Radeon 8060S Graphics - Ubuntu 24.04, Vulkan llama.cpp via lmstudio)
Enabling flash attention creates a crash - a different one, but a crash nonetheless. Here tested with context size 20096 which works well without flash attention:
error as displayed in an error popup in LMstudio UI:
the llama.cpp log (which does not appear to contain any error):
illioren commentedon Aug 7, 2025
The max context size seems to be 31520 (for both 120b and 20b models, 31521 crashes, 31520 works...)
No idea if this is significant... but it is 1248 less than the 32768 (32k) models limit (4*8192).
Hopefully, that helps troubleshooting in some ways :)
Mushoz commentedon Aug 7, 2025
With
-faI can now load the full context0cc4m commentedon Aug 10, 2025
If your problem is resolved, please close the issue.
Mushoz commentedon Aug 10, 2025
I mean I am more than happy to close it, as it doesn't really impact me since I will run with flash attention if possible. But isn't this still a bug? It should be possible to run without FA?
10 remaining items
0cc4m commentedon Sep 15, 2025
Oh yeah, I didn't see that. I guess the queue submission needed a little bit of memory and there was no usable space left, for whatever reason.
Try only allocating a small dedicated part of the RAM to the iGPU and leaving the rest shared, the Vulkan backend should still be able to use it. Maybe it works that way.
slaren commentedon Sep 15, 2025
Does it work with
--no-mmap?apkrieg commentedon Sep 15, 2025
Sure thing, there it is: https://gist.github.com/apkrieg/5f8406561cc3c78b4f4b0300f347e8bb
Just tried this. Set it to the default setting of 512MB, but it seems to them only be sharing 64GB from system memory and loading the models fails as before.
The command I used includes
--no-mmap:./llama-cli -ngl 999 -dev Vulkan0 --no-mmap -m "C:\Users\Andrew\.lmstudio\models\bartowski\Monstral-123B-v2-GGUF\Monstral-123B-v2-Q4_K_M-00001-of-00002.gguf"0cc4m commentedon Sep 16, 2025
I think there are ways to increase that share: https://github.com/lhl/strix-halo-testing/tree/main/llm-bench#gpu-memory , but I haven't tried this.
Edit: The way described there is deprecated, here is a description of the recommended way to do that: https://www.jeffgeerling.com/blog/2025/increasing-vram-allocation-on-amd-ai-apus-under-linux
apkrieg commentedon Sep 16, 2025
Unfortunately I'm running Windows 11
0cc4m commentedon Sep 16, 2025
Unfortunate, then that's not possible without swapping OS. I don't think we can do more from here, it also sounds like a driver bug to me.
arun-mani-j commentedon Sep 27, 2025
I'm having the same error but with
gpt-oss-20b-mxfp4.gguf. I have an integrated AMD GPU (the backend in usage is Vulkan), so don't think I can expect much :(I have
22 Giof total RAM. So is this error a limitation of my system?Command:
0cc4m commentedon Nov 9, 2025
Can you check if this still happens? #17110 may have helped.
apkrieg commentedon Nov 9, 2025
I can confirm I've been consistently loading very large models without issue. Thank you very much
EDIT: It's using my entire dedicated 96GB pool (awesome), but I don't think it's allocating anything from the additionally available 16GB shared memory. Not sure if that's expected or not
Uglevod commentedon Nov 20, 2025
I have micro hardware for testing: an RX580 8GB graphics card, 2GB DDR2 RAM, and an AMD Athlon 64 X2 6400 processor. I'm running Xubuntu 22.04 and trying to use small language models (LMMs) ranging from 600MB to 3.9GB in size.
When I try to load the models on this system, the swap space was not used, which results in an Out of Memory (OOM) error. However, when I run the same setup in a Proxmox virtual machine with an Intel Xeon v4 2680 processor ( and RX580 ), I can successfully load the 3.9GB model to VRAM even with only 2GB of RAM allocated with using swap space.
github-actions commentedon Jan 5, 2026
This issue was closed because it has been inactive for 14 days since being marked as stale.