Repository navigation
Eval bug: When I run Gemma 4e4b with MTP, it doesn't work and crashes to cmd #26017
Copy link
Copy link
Closed as not planned
Closed as not planned
Copy link
Labels
Description
Name and Version
I'm using llama.cpp b10090.
Operating systems
Windows
GGML backends
CUDA
Hardware
5 5600g and gtx 1660 super
Models
gemma-4-E4B-it-UD-Q4_K_XL and mtp-gemma-4-E4B-it
Problem description & steps to reproduce
it doesn't work with either CUDA 12 or 13 but vulkan work. I'm using this configuration: llama-cpp\llama-server.exe -m gemma-4-e4b\gemma-4-E4B-it-UD-Q4_K_XL.gguf -md gemma-4-e4b\mtp-gemma-4-E4B-it.gguf --fit off --reasoning on --n-gpu-layers all --ctx-size 131072 --spec-type draft-mtp --spec-draft-n-max 2. I've checked it several times, but there doesn't seem to be a problem with my configuration. I just downloaded those two models and applied that configuration.
First Bad Commit
I got an error when testing those models for the first time in version b10088
Relevant log output
Logs
0.00.100.481 I cmn common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.175.670 W srv llama_server: ----------------- 0.00.175.675 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.175.676 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.175.676 W srv llama_server: more info: https://github.com//pull/25655 0.00.175.676 W srv llama_server: ----------------- 0.00.190.688 I srv load_model: loading model 'gemma-4-e4b\gemma-4-E4B-it-UD-Q4_K_XL.gguf' 0.00.786.499 W load: control-looking token: 1 '' was not control-type; this is probably a bug in the model. its type will be overridden 0.00.786.828 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden 0.00.787.301 W load: control-looking token: 212 '' was not control-type; this is probably a bug in the model. its type will be overridden 0.00.840.128 W load: special_eog_ids contains '<|tool_response>', removing '' token from EOG list 0.16.151.181 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden 0.16.151.676 W load: control-looking token: 212 '' was not control-type; this is probably a bug in the model. its type will be overridden 0.16.200.384 W load: special_eog_ids contains '<|tool_response>', removing '' token from EOG list 0.16.291.760 E llama_model_load: error loading model: invalid vector subscript 0.16.291.768 E llama_model_load_from_file_impl: failed to load model 0.16.291.770 E common_speculative_init_result: failed to load draft model, 'gemma-4-e4b\mtp-gemma-4-E4B-it.gguf' 0.16.291.773 E srv load_model: failed to load draft model, 'gemma-4-e4b\mtp-gemma-4-E4B-it.gguf' 0.16.291.799 I srv operator(): operator(): cleaning up before exit... 0.16.292.800 E srv llama_server: exiting due to model loading errorReactions are currently unavailable
Activity
felipemolinafacundo-del commentedon Jul 23, 2026
Update: I just quantized the cache to q8_0 and it worked; in case you're wondering, it did work without MTP using the full cache. Also, I have 16 GB of RAM.
github-actions commentedon Sep 6, 2026
This issue was closed because it has been inactive for 14 days since being marked as stale.