Skip to content

Eval bug: When I run Gemma 4e4b with MTP, it doesn't work and crashes to cmd #26017

Closed as not planned

Description

Name and Version

I'm using llama.cpp b10090.

Operating systems

Windows

GGML backends

CUDA

Hardware

5 5600g and gtx 1660 super

Models

gemma-4-E4B-it-UD-Q4_K_XL and mtp-gemma-4-E4B-it

Problem description & steps to reproduce

it doesn't work with either CUDA 12 or 13 but vulkan work. I'm using this configuration: llama-cpp\llama-server.exe -m gemma-4-e4b\gemma-4-E4B-it-UD-Q4_K_XL.gguf -md gemma-4-e4b\mtp-gemma-4-E4B-it.gguf --fit off --reasoning on --n-gpu-layers all --ctx-size 131072 --spec-type draft-mtp --spec-draft-n-max 2. I've checked it several times, but there doesn't seem to be a problem with my configuration. I just downloaded those two models and applied that configuration.

First Bad Commit

I got an error when testing those models for the first time in version b10088

Relevant log output

Logs 0.00.100.481 I cmn common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.175.670 W srv llama_server: ----------------- 0.00.175.675 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.175.676 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.175.676 W srv llama_server: more info: https://github.com//pull/25655 0.00.175.676 W srv llama_server: ----------------- 0.00.190.688 I srv load_model: loading model 'gemma-4-e4b\gemma-4-E4B-it-UD-Q4_K_XL.gguf' 0.00.786.499 W load: control-looking token: 1 '' was not control-type; this is probably a bug in the model. its type will be overridden 0.00.786.828 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden 0.00.787.301 W load: control-looking token: 212 '' was not control-type; this is probably a bug in the model. its type will be overridden 0.00.840.128 W load: special_eog_ids contains '<|tool_response>', removing '' token from EOG list 0.16.151.181 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden 0.16.151.676 W load: control-looking token: 212 '' was not control-type; this is probably a bug in the model. its type will be overridden 0.16.200.384 W load: special_eog_ids contains '<|tool_response>', removing '' token from EOG list 0.16.291.760 E llama_model_load: error loading model: invalid vector subscript 0.16.291.768 E llama_model_load_from_file_impl: failed to load model 0.16.291.770 E common_speculative_init_result: failed to load draft model, 'gemma-4-e4b\mtp-gemma-4-E4B-it.gguf' 0.16.291.773 E srv load_model: failed to load draft model, 'gemma-4-e4b\mtp-gemma-4-E4B-it.gguf' 0.16.291.799 I srv operator(): operator(): cleaning up before exit... 0.16.292.800 E srv llama_server: exiting due to model loading error

Activity

  1. felipemolinafacundo-del commented on Jul 23, 2026

    @felipemolinafacundo-del
    Author

    Update: I just quantized the cache to q8_0 and it worked; in case you're wondering, it did work without MTP using the full cache. Also, I have 16 GB of RAM.

  2. github-actions commented on Sep 6, 2026

    @github-actions
    Contributor

    This issue was closed because it has been inactive for 14 days since being marked as stale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions