Misc. bug: llama-server --models-preset creates an unexpected "default" model entry

[vanbukin]

Name and Version

root@pc2:~# /opt/llama.cpp/bin/llama-server --version
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24079 MiB):
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes, VRAM: 24079 MiB
version: 8931 (9725a313b)
built with GNU 15.2.0 for Linux x86_64

root@pc2:~# cat /etc/os-release
PRETTY_NAME="Ubuntu 26.04 LTS"
NAME="Ubuntu"
VERSION_ID="26.04"
VERSION="26.04 (Resolute Raccoon)"
VERSION_CODENAME=resolute
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=resolute
LOGO=ubuntu-logo

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

/opt/llama.cpp/bin/llama-server \
--host 127.0.0.1 \
--port 8080 \
--models-preset /etc/llama.cpp/models.ini \
--models-max 1 \
--models-autoload \
--api-key-file /etc/llama.cpp/api_keys \
--metrics

Problem description & steps to reproduce

When running llama-server in router mode with --models-preset, an extra model with id default appears in GET /models, even though there is no [default] section in the preset file and no model is specified on the command line.

According to the documentation, each section in the INI file defines a new preset. In my case, the INI file contains only two model sections, but the server reports three custom presets and exposes a third default model.

Preset file

/etc/llama.cpp/models.ini:

version = 1

[qwen3.6-27b]
model = /srv/llama.cpp/models/unsloth/Qwen3.6-27B-GGUF/Qwen3.6-27B-UD-Q4_K_XL.gguf
mmproj = /srv/llama.cpp/models/unsloth/Qwen3.6-27B-GGUF/mmproj-BF16.gguf
ctx-size = 122880
ubatch-size = 1024
batch-size = 122880
n-cpu-moe = 0
n-gpu-layers = 999
temp = 0.6
top-p = 0.95
top-k = 20
min-p = 0.00
repeat-penalty = 1.0
presence-penalty = 0.0
split-mode = none
main-gpu = 0
image-min-tokens = 1024
image-max-tokens = 2048
parallel = 1
flash-attn = on
cache-type-k = q8_0
cache-type-v = q8_0
chat-template-kwargs = {"enable_thinking":true}
load-on-startup = true
stop-timeout = 10

[qwen3-asr-1.7b]
model = /srv/llama.cpp/models/ggml-org/Qwen3-ASR-1.7B-GGUF/Qwen3-ASR-1.7B-bf16.gguf
mmproj = /srv/llama.cpp/models/ggml-org/Qwen3-ASR-1.7B-GGUF/mmproj-Qwen3-ASR-1.7B-bf16.gguf
ctx-size = 65536
ubatch-size = 2048
batch-size = 65536
n-cpu-moe = 0
n-gpu-layers = 999
split-mode = none
main-gpu = 0
parallel = 1
load-on-startup = false
stop-timeout = 10

Steps to reproduce

1. Create a model preset file with two model sections and no [default] section, as shown above.
2. Start llama-server in router mode:

/opt/llama.cpp/bin/llama-server \
--host 127.0.0.1 \
--port 8080 \
--models-preset /etc/llama.cpp/models.ini \
--models-max 1 \
--models-autoload \
--api-key-file /etc/llama.cpp/api_keys \
--metrics

3. Query the models endpoint:

curl http://127.0.0.1:8080/models

Expected behavior

Only the two models explicitly defined in the preset file should be listed:

• qwen3-asr-1.7b
• qwen3.6-27b

The startup log should also report 2 custom model presets loaded from /etc/llama.cpp/models.ini.

Actual behavior

The server reports 3 custom model presets and exposes an unexpected third model named default

{
    "data": [
        {
            "id": "default",
            "aliases": [],
            "tags": [],
            "object": "model",
            "owned_by": "llamacpp",
            "created": 1777137717,
            "status": {
                "value": "unloaded",
                "args": [
                    "/opt/llama.cpp/bin/llama-server",
                    "--api-key-file",
                    "/etc/llama.cpp/api_keys",
                    "--host",
                    "127.0.0.1",
                    "--metrics",
                    "--port",
                    "0",
                    "--alias",
                    "default"
                ],
                "preset": "[default]\napi-key-file = /etc/llama.cpp/api_keys\nmetrics = 1\n\n"
            }
        },
        {
            "id": "qwen3-asr-1.7b",
            "aliases": [],
            "tags": [],
            "object": "model",
            "owned_by": "llamacpp",
            "created": 1777137717,
            "status": {
                "value": "unloaded",
                "args": [
                    "/opt/llama.cpp/bin/llama-server",
                    "--api-key-file",
                    "/etc/llama.cpp/api_keys",
                    "--host",
                    "127.0.0.1",
                    "--metrics",
                    "--port",
                    "0",
                    "--alias",
                    "qwen3-asr-1.7b",
                    "--batch-size",
                    "65536",
                    "--ctx-size",
                    "65536",
                    "--model",
                    "/srv/llama.cpp/models/ggml-org/Qwen3-ASR-1.7B-GGUF/Qwen3-ASR-1.7B-bf16.gguf",
                    "--main-gpu",
                    "0",
                    "--mmproj",
                    "/srv/llama.cpp/models/ggml-org/Qwen3-ASR-1.7B-GGUF/mmproj-Qwen3-ASR-1.7B-bf16.gguf",
                    "--n-cpu-moe",
                    "0",
                    "--n-gpu-layers",
                    "999",
                    "--parallel",
                    "1",
                    "--split-mode",
                    "none",
                    "--ubatch-size",
                    "2048"
                ],
                "preset": "[qwen3-asr-1.7b]\napi-key-file = /etc/llama.cpp/api_keys\nmetrics = 1\nbatch-size = 65536\nctx-size = 65536\nmodel = /srv/llama.cpp/models/ggml-org/Qwen3-ASR-1.7B-GGUF/Qwen3-ASR-1.7B-bf16.gguf\nmain-gpu = 0\nmmproj = /srv/llama.cpp/models/ggml-org/Qwen3-ASR-1.7B-GGUF/mmproj-Qwen3-ASR-1.7B-bf16.gguf\nn-cpu-moe = 0\nn-gpu-layers = 999\nparallel = 1\nsplit-mode = none\nubatch-size = 2048\nload-on-startup = false\nstop-timeout = 10\n\n"
            }
        },
        {
            "id": "qwen3.6-27b",
            "aliases": [],
            "tags": [],
            "object": "model",
            "owned_by": "llamacpp",
            "created": 1777137717,
            "status": {
                "value": "loaded",
                "args": [
                    "/opt/llama.cpp/bin/llama-server",
                    "--api-key-file",
                    "/etc/llama.cpp/api_keys",
                    "--chat-template-kwargs",
                    "{\"enable_thinking\":true}",
                    "--host",
                    "127.0.0.1",
                    "--image-max-tokens",
                    "2048",
                    "--image-min-tokens",
                    "1024",
                    "--metrics",
                    "--min-p",
                    "0.00",
                    "--port",
                    "53891",
                    "--presence-penalty",
                    "0.0",
                    "--repeat-penalty",
                    "1.0",
                    "--temperature",
                    "0.6",
                    "--top-k",
                    "20",
                    "--top-p",
                    "0.95",
                    "--alias",
                    "qwen3.6-27b",
                    "--batch-size",
                    "122880",
                    "--ctx-size",
                    "122880",
                    "--cache-type-k",
                    "q8_0",
                    "--cache-type-v",
                    "q8_0",
                    "--flash-attn",
                    "on",
                    "--model",
                    "/srv/llama.cpp/models/unsloth/Qwen3.6-27B-GGUF/Qwen3.6-27B-UD-Q4_K_XL.gguf",
                    "--main-gpu",
                    "0",
                    "--mmproj",
                    "/srv/llama.cpp/models/unsloth/Qwen3.6-27B-GGUF/mmproj-BF16.gguf",
                    "--n-cpu-moe",
                    "0",
                    "--n-gpu-layers",
                    "999",
                    "--parallel",
                    "1",
                    "--split-mode",
                    "none",
                    "--ubatch-size",
                    "1024"
                ],
                "preset": "[qwen3.6-27b]\napi-key-file = /etc/llama.cpp/api_keys\nchat-template-kwargs = {\"enable_thinking\":true}\nimage-max-tokens = 2048\nimage-min-tokens = 1024\nmetrics = 1\nmin-p = 0.00\npresence-penalty = 0.0\nrepeat-penalty = 1.0\ntemperature = 0.6\ntop-k = 20\ntop-p = 0.95\nbatch-size = 122880\nctx-size = 122880\ncache-type-k = q8_0\ncache-type-v = q8_0\nflash-attn = on\nmodel = /srv/llama.cpp/models/unsloth/Qwen3.6-27B-GGUF/Qwen3.6-27B-UD-Q4_K_XL.gguf\nmain-gpu = 0\nmmproj = /srv/llama.cpp/models/unsloth/Qwen3.6-27B-GGUF/mmproj-BF16.gguf\nn-cpu-moe = 0\nn-gpu-layers = 999\nparallel = 1\nsplit-mode = none\nubatch-size = 1024\nload-on-startup = true\nstop-timeout = 10\n\n"
            }
        }
    ],
    "object": "list"
}

The unexpected default entry is not defined in the INI file and does not have a model path.

This is confusing for clients that use GET /models to discover available models, because the endpoint returns a model that was never configured and cannot be used as a real preset.

First Bad Commit

No response

Relevant log output

Startup log

Logs

root@pc2:~# /opt/llama.cpp/bin/llama-server --host 127.0.0.1 --port 8080 --models-preset /etc/llama.cpp/models.ini --models-max 1 --models-autoload --api-key-file /etc/llama.cpp/api_keys --metrics
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24079 MiB):
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes, VRAM: 24079 MiB
main: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
build_info: b8931-9725a313b
system_info: n_threads = 16 (n_threads_batch = 16) / 32 | CUDA : ARCHS = 890 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
Running without SSL
init: api_keys: ****42a8
init: using 31 threads for HTTP server
srv   load_models: Loaded 0 cached model presets
srv   load_models: Loaded 3 custom model presets from /etc/llama.cpp/models.ini
srv   load_models: Available models (3) (*: custom preset)
srv   load_models:   * default
srv   load_models:   * qwen3-asr-1.7b
srv   load_models:   * qwen3.6-27b
srv   load_models: (startup) loading model qwen3.6-27b
srv          load: spawning server instance with name=qwen3.6-27b on port 53891
...

[naumf]

I stumbled upon the same issue and removing the version and the empty line below it from the .ini file fixed it

version = 1

[jsdnrs]

To add context to this issue, removing the version key, as @naumf recommends, shown in the example of https://github.com/ggml-org/llama.cpp/tree/master/tools/server#model-presets corrects this behavior and shouldn't cause any issues as its just skipped over:

llama.cpp/common/preset.cpp

Lines 323 to 326 in b97ebdc

	if (key == "version") {
	// skip version key (reserved for future use)
	continue;
	}

Whether or not this is a bug would be up to @ngxson's original intentions in #17859.

The behavior comes from the load_from_ini() function:

llama.cpp/common/preset.cpp

Lines 310 to 360 in b97ebdc

	common_presets common_preset_context::load_from_ini(const std::string & path, common_preset & global) const {
	common_presets out;
	auto ini_data = parse_ini_from_file(path);
	for (auto section : ini_data) {
	common_preset preset;
	if (section.first.empty()) {
	preset.name = COMMON_PRESET_DEFAULT_NAME;
	} else {
	preset.name = section.first;
	}
	LOG_DBG("loading preset: %s\n", preset.name.c_str());
	for (const auto & [key, value] : section.second) {
	if (key == "version") {
	// skip version key (reserved for future use)
	continue;
	}
	LOG_DBG("option: %s = %s\n", key.c_str(), value.c_str());
	if (filter_allowed_keys && allowed_keys.find(key) == allowed_keys.end()) {
	throw std::runtime_error(string_format(
	"option '%s' is not allowed in remote presets",
	key.c_str()
	));
	}
	if (key_to_opt.find(key) != key_to_opt.end()) {
	const auto & opt = key_to_opt.at(key);
	if (is_bool_arg(opt)) {
	preset.options[opt] = parse_bool_arg(opt, key, value);
	} else {
	preset.options[opt] = value;
	}
	LOG_DBG("accepted option: %s = %s\n", key.c_str(), preset.options[opt].c_str());
	} else {
	throw std::runtime_error(string_format(
	"option '%s' not recognized in preset '%s'",
	key.c_str(), preset.name.c_str()
	));
	}
	}
	if (preset.name == "*") {
	// handle global preset
	global = preset;
	} else {
	out[preset.name] = preset;
	}
	}
	return out;
	}

As part of parsing the INI-style file, the function must assume a first section, which, if not yet defined, becomes COMMON_PRESET_DEFAULT_NAME="default" from common/preset.h. Then, it assigns the key-value pairs in the section, skipping the version key as mentioned above. After going through all the key-value pairs in the section, it creates a new mapping of the section name and key-value pairs, which if only including the version key as shown in the example from the llama-server documentation results in an empty "default" mapping.

This "default" section, which isn't mentioned in the documentation, is notably separate from the global "*" section, which is mentioned in https://github.com/ggml-org/llama.cpp/tree/master/tools/server#model-presets. I believe both are necessary as they serve unique functions.

If this isn't the intended behavior, there are plenty of ways to correct it. Such as, adding a check if preset.name == "default", same as done for "*" on L351 above. Then either just skip it, store it for later, or don't publish it at the /models endpoint.

If it is the intended behavior, possibly just mention it in the documentation at: https://github.com/ggml-org/llama.cpp/tree/master/tools/server#model-presets

Love llama.cpp's router mode by the way!

[hogeheer499-commits]

I reproduced this on current master (bbeb89d76) with the documented version = 1 line before the first section.

Preset used:

version = 1

[custom_model]
model = /models/Qwen_Qwen3-0.6B-Q8_0.gguf

Started the router with:

llama-server --models-preset /tmp/models.ini --no-models-autoload --no-webui --host 127.0.0.1 --port 18080

/v1/models returns both entries:

custom_model
default

As a control, removing only the top-level version = 1 line makes /v1/models return only:

custom_model

The server log also shows the difference:

Loaded 2 custom model presets from /tmp/models.ini
  * custom_model
  * default

Loaded 1 custom model presets from /tmp/models-no-version.ini
  * custom_model

So this reproduces with the README-style preset file, not just by inspecting the parser.

Disclosure: AI was used for note organization only.

[hogeheer499-commits]

I opened a small fix for this in #22741.

The cause was that version = 1 is parsed into the implicit default section before the first named section. Since version is metadata and gets skipped, the implicit default preset was left empty but still returned as a custom preset.

The patch skips only empty implicit default presets, while keeping real default presets and [*] global presets covered by tests.

[vijaygill]

Ha ha I was bitten by this just a few minutes ago and I was wondering where this "default" is coming from!

Waiting for the fix to be merged.

Thanks everyone for the good work.