Eval bug: DeepSeek V4 Flash tokenizer blows its stack on long tool output #26965

Name and Version

version: 10293 (a1f96d4)
built with GNU 13.3.0 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

1x NVIDIA RTX PRO 6000 Blackwell Server Edition, 97,887 MiB VRAM

Models

DeepSeek V4 Flash 0731, UD-IQ1_S

https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF/tree/1290dcca3f84612f646fb546fb9e8433c1b339b0/UD-IQ1_S

Problem description & steps to reproduce

Hit this while running DeepSeek V4 Flash 0731 through llama-server.

A large tool result containing one long run of ASCII letters crashes the server during tokenization, before inference gets a turn.

Steps to reproduce

1. Check out llama.cpp commit a1f96d4fc2c9e4101a6666a9d87f547e7e880df6.

2. Build with CUDA and start the server:

llama-server \
  -m DeepSeek-V4-Flash-0731-UD-IQ1_S-00001-of-00003.gguf \
  -c 16384 \
  -b 2048 \
  -ub 512 \
  -np 1 \
  --swa-full \
  -fa on \
  -ctk f16 \
  -ctv f16 \
  -ngl 999 \
  --host 127.0.0.1 \
  --port 18082 \
  --jinja \
  --reasoning-format deepseek \
  --reasoning on \
  --reasoning-budget -1

3. Run an OpenAI-compatible tool-use conversation against /v1/chat/completions.

4. Have one tool produce a long letter run:

python3 -c "print('Z' * 131072)"

5. Send the tool result back to the model. In the tested run, Pi 0.84.0 clipped it to 51,294 bytes.

Server falls over with SIGSEGV while tokenizing that request.

What should happen

If the request is too large, return a normal context limit error and keep the server alive.

What actually happens

libstdc++ piles up roughly 76,000 recursive std::regex frames and exhausts the worker thread stack. Backtrace ends up here:

std::__detail::__regex_algo_impl
unicode_regex_split
llm_tokenizer_bpe_session::tokenize
llama_vocab::tokenize
common_tokenize
tokenize_input_prompts
server_routes::handle_completions_impl

GGUF has tokenizer.ggml.pre set to joyai-llm. That is just the pre-tokenizer compatibility tag. Its regex is also used by DeepSeek V3 and Hunyuan Dense.

No custom splitter handles this exact regex, so unicode_regex_split() punts it to std::regex. Long letter runs then chew through the thread stack.

A non-recursive splitter fixes the crash. Replaying the original request returns the expected 27940 tokens exceeds 16384 tokens error, and /health stays ok.

First Bad Commit

No response

Relevant log output

  CUDA Version 12.8.1

  Thread 26 "llama-server" received signal SIGSEGV, Segmentation fault.
  [Switching to Thread 0x7ffd557f2000 (LWP 155)]

  #0      std::__detail::_Executor<...>::_M_dfs(...)
  #1      std::__detail::_Executor<...>::_M_rep_once_more(...)
          ... same two std::regex frames repeat about 76,000 times ...
  #76664  std::__detail::__regex_algo_impl(...)
  #76665  unicode_regex_split(...)
  #76666  llm_tokenizer_bpe_session::tokenize(...)
  #76667  llama_vocab::impl::tokenize(...)
  #76668  llama_vocab::tokenize(...)
  #76669  llama_vocab::tokenize(...)
  #76670  common_tokenize(...)
  #76671  tokenize_mixed(...)
  #76672  tokenize_input_subprompt(...)
  #76673  tokenize_input_prompts(...)
  #76674  server_routes::handle_completions_impl(...)