[Feature]Support tag phase token enforce generation - #6031
Merged
yuanlehome merged 4 commits intoJan 15, 2026
Merged
yuanlehome merged 4 commits into
yuanlehome merged 4 commits into
Conversation
|
Thanks for your contribution! |
heavengate
reviewed
Jan 14, 2026
| @@ -0,0 +1,305 @@ | |||
| // Copyright (c) 2024 PaddlePaddle Authors | |||
Contributor
There was a problem hiding this comment.
Pull request overview
本PR实现了一个推理阶段的token约束生成功能，用于在模型生成\n</think>\n\n模式后，强制下一个token必须是<tool_call>或<response>中的一个。
Changes:
- 新增CUDA kernel
reasoning_phase_token_constraint.cu实现状态机逻辑和logits约束 - 在SpeculativeSampler中集成该功能，通过
enf_gen_phase_tag配置项控制 - 修改
speculate_verify以支持reasoning_status参数，在状态1时限制draft token接受 - 添加完整的单元测试覆盖状态转换和logits约束逻辑
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| custom_ops/gpu_ops/reasoning_phase_token_constraint.cu | 新增CUDA kernel实现推理阶段状态机和token约束逻辑 |
| fastdeploy/model_executor/layers/sample/sampler.py | 在SpeculativeSampler中集成reasoning token约束功能 |
| fastdeploy/model_executor/layers/sample/ops/apply_penalty_multi_scores.py | 添加reasoning_phase_token_constraint包装函数 |
| fastdeploy/model_executor/layers/sample/ops/init.py | 导出新函数 |
| fastdeploy/config.py | 添加enf_gen_phase_tag配置项 |
| custom_ops/gpu_ops/speculate_decoding/speculate_verify.cu | 添加reasoning_status参数支持 |
| tests/operators/test_reasoning_phase_token_constraint.py | 新增完整的单元测试 |
| tests/operators/test_speculate_verify.py | 更新测试以支持新参数 |
| custom_ops/setup_ops.py | 添加新的CUDA文件到构建 |
| custom_ops/gpu_ops/cpp_extensions.cc | 添加C++绑定 |
Comment on lines
+154
to
+218
| def test_status_0_to_1_only(self): | ||
| """ | ||
| status == 0 | ||
| recent tokens contain <think_end> | ||
| => status: 0 -> 1 | ||
| logits should NOT be enforced | ||
| """ | ||
|
|
||
| # ------------------------ | ||
| # setup: only think_end appears | ||
| # ------------------------ | ||
| pre_ids = np.zeros((self.bs, self.max_seq_len), dtype=np.int64) | ||
|
|
||
| # batch 0: think_end at cur_step - 1 | ||
| pre_ids[0, 3] = self.think_end_id | ||
|
|
||
| # batch 1: no think_end | ||
| pre_ids[1, :] = 0 | ||
|
|
||
| self.pre_ids = paddle.to_tensor(pre_ids, dtype="int64") | ||
|
|
||
| self.reasoning_status = paddle.to_tensor([0, 0], dtype="int32") | ||
|
|
||
| logits_before = self.logits.numpy().copy() | ||
|
|
||
| # ------------------------ | ||
| # call op | ||
| # ------------------------ | ||
| reasoning_phase_token_constraint( | ||
| self.logits, | ||
| self.pre_ids, | ||
| self.stop_flags, | ||
| self.seq_lens_this_time, | ||
| self.seq_lens_encoder, | ||
| self.step_idx, | ||
| self.allowed_tokens, | ||
| self.reasoning_status, | ||
| self.output_padding_offset, | ||
| self.output_cum_offsets, | ||
| self.think_end_id, | ||
| self.line_break_id, | ||
| ) | ||
|
|
||
| status_after = self.reasoning_status.numpy() | ||
| logits_after = self.logits.numpy() | ||
|
|
||
| # ============================================================ | ||
| # 1. reasoning_status | ||
| # ============================================================ | ||
| # batch 0: 0 -> 1 | ||
| self.assertEqual(status_after[0], 1) | ||
|
|
||
| # batch 1: stays 0 | ||
| self.assertEqual(status_after[1], 0) | ||
|
|
||
| # ============================================================ | ||
| # 2. logits must be untouched | ||
| # ============================================================ | ||
| np.testing.assert_allclose( | ||
| logits_after, | ||
| logits_before, | ||
| rtol=1e-5, | ||
| atol=1e-6, | ||
| ) | ||
|
|
There was a problem hiding this comment.
测试用例缺少对cur_step < 3场景的覆盖。所有测试用例都使用step_idx = 4，这无法触发状态机在cur_step < 3时的状态转换bug。
建议添加测试用例：
- status == 0，cur_step = 2，检测到think_end_id
- 预期：status应该转换为1
- 实际：由于bug，status不会被更新
这个测试用例对于验证状态机的完整性很重要。
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## release/online/20251131 #6031 +/- ##
==========================================================
Coverage ? 56.80%
==========================================================
Files ? 324
Lines ? 39388
Branches ? 5944
==========================================================
Hits ? 22374
Misses ? 15206
Partials ? 1808
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
EmmonsCurse
had a problem deploying
to
Metax_ci
January 14, 2026 11:07 — with 
GitHub Actions
Failure
GitHub Actions
Failure
yuanlehome
merged commit Jan 15, 2026
c8e882f
into
PaddlePaddle:release/online/20251131
11 of 16 checks passed
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
\n</think>\n\n后，生成<tool_call>/<response>其中一个tokenModifications
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.