Skip to content

Fix llama.cpp chat token ID parity - #143

Merged
Qubitium merged 3 commits into
mainfrom
fix/llamacpp-chat-token-ids
Sep 4, 2026
Merged

Fix llama.cpp chat token ID parity#143
Qubitium merged 3 commits into
mainfrom
fix/llamacpp-chat-token-ids

Conversation

@Qubitium

@Qubitium Qubitium commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • pass HF-rendered chat token IDs directly to llama.cpp
  • avoid re-tokenizing rendered special-token text or duplicating BOS
  • normalize BatchEncoding/tensor/list chat-template outputs

Validation

  • 24 focused engine tests pass
  • real Llama 3.2 tokenizer telemetry shows HF, Evalution, and llama.cpp chat token IDs have identical SHA-256
  • official llama.cpp GGUF sanity generation returns the expected answer

@Qubitium
Qubitium merged commit 8cbed6e into main Sep 4, 2026
2 checks passed
@Qubitium
Qubitium deleted the fix/llamacpp-chat-token-ids branch September 4, 2026 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant