Background
Dream-7B is a 7B-parameter diffusion language model from the Dream team (HKUST + Huawei). Architecturally similar to LLaDA - masked diffusion, bidirectional attention, parallel commit - but uses a different commit policy: entropy-based rather than confidence top-k.
Investigation during v0.1 development found that the default top-k confidence commit causes Dream to emit empty strings (content: "") when max_tokens ≥ 256. The model's confidence distribution doesn't match LLaDA's: high-entropy positions dominate and the top-k selection picks EOS cascades. At max_tokens ≤ 128, temperature=0, Dream works correctly.
Acceptance criteria
References
Background
Dream-7B is a 7B-parameter diffusion language model from the Dream team (HKUST + Huawei). Architecturally similar to LLaDA - masked diffusion, bidirectional attention, parallel commit - but uses a different commit policy: entropy-based rather than confidence top-k.
Investigation during v0.1 development found that the default top-k confidence commit causes Dream to emit empty strings (
content: "") whenmax_tokens ≥ 256. The model's confidence distribution doesn't match LLaDA's: high-entropy positions dominate and the top-k selection picks EOS cascades. Atmax_tokens ≤ 128, temperature=0, Dream works correctly.Acceptance criteria
dlmserve/models/dream.pyloader (analogous tollada.py)commit_top_k_by_negative_entropy()insampler.pydlmserve/engine.py)✓ v0.1.1examples/chat.py --model Dream-org/Dream-v0-Instruct-7Bproduces coherent output atmax_tokens=256References
docs/adrs/for the EOS-cascade analysis