Repository navigation
Conversation
…cefully Drafters that borrow tensors from the target model (e.g. DFlash models without their own token embeddings / output head) require ctx_other at context creation. The example creates the draft context without it, so llama_init_from_model returns null, and the unchecked ctx_dft is later passed to llama_decode, which crashes. Pass the target context as ctx_other, error out cleanly if the draft context still cannot be created, and warn that this example does not stage target-model features for dflash drafters (acceptance will be much lower than with llama-server, which uses the full framework path).
|
Hi @bri-prism, thanks for your contribution! Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:
Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below. |
ggml-gh-bot
Bot
added
the
draft
github-actions
Bot
removed
the
draft
bri-prism
marked this pull request as ready for review
August 11, 2026 05:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
llama-speculative-simplecreates the draft context withoutctx_otherand never null-checks the result. For drafters that borrow tensors from the target model (DFlash models without their own token embeddings or output head, such as the deepseek-ai dspark checkpoints),llama_init_from_modelreturns null, and the uncheckedctx_dftreachesllama_decode, which crashes.This PR makes the following changes:
ctx_otherwhen creating the draft context.llama-server, which uses the full framework path.Additional information
The crash can be reproduced with:
The failure occurs at prompt evaluation. The "dflash requires ctx_other" message emitted during context creation is easy to miss, because the same message also appears benignly during memory fitting.
The feature-staging gap itself is described in the companion issue #26884. I am happy to work on full DFlash support in this example if there is interest.
Requirements