Repository navigation
feat(vllm_model): add opt-in fold_nonleading_system_messages - #4010
Open
linj-glitch wants to merge 3 commits into
Open
linj-glitch wants to merge 3 commits into
linj-glitch wants to merge 3 commits into
Conversation
yaoyu-33
added
area:model
linj-glitch
marked this pull request as draft
October 5, 2026 06:12
linj-glitch
force-pushed
the
linj/vllm-fold-system-messages
branch
from
October 8, 2026 00:15
9eae98e to
a396650
Compare
Some harnesses inject system messages mid-conversation; Claude Code, for example, appends background-task notices with role=system after tool turns. Chat templates that only admit a system message at the start (e.g. "System message must be at the beginning") reject the whole request, which the harness sees as a server error. Add `fold_nonleading_system_messages` to VLLMModelConfig (default off). When set, every system/developer message after the leading system block is sent as a user message with identical content, so the model still reads it in chronological position. It runs after `replace_developer_role_with_system` and, like that option, raises NotImplementedError on the Responses-native path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Lin Jia <linj@nvidia.com>
linj-glitch
force-pushed
the
linj/vllm-fold-system-messages
branch
from
October 8, 2026 00:20
a396650 to
d42f056
Compare
linj-glitch
marked this pull request as ready for review
October 13, 2026 23:08
Signed-off-by: Lin Jia <linj@nvidia.com>
github-actions
Bot
added
the
sla:triage-overdue
This was referenced Oct 22, 2026
Collaborator
Author
|
/ok to test 6c73d74 |
Contributor
|
🌿 Preview your docs: https://nvidia-preview-pr-4010-8ce785af39a5.docs.buildwithfern.com/nemo/gym |
Signed-off-by: Lin Jia <linj@nvidia.com>
Collaborator
Author
|
/ok to test 8ce785a |
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Claude Code can insert a system notice after a tool turn, which causes Qwen-style chat templates to reject the request with
System message must be at the beginning.Enabling
fold_nonleading_system_messageschanges later system/developer messages to user messages while preserving their content, chronological position, and the leading system block, after applyingreplace_developer_role_with_system.The option defaults to false and rejects the Responses-native path; enabling it deliberately changes those prompt roles.
Head
8ce785af39a50ca2e63cb025a8b424243b2ecd5cincludes an additive DCO-signed merge of mainf36e126547866601bee9c0671c9b59484ef83723to pick up the existing mini-SWE dependency-isolation and terminal-test fixes, with the folding behavior unchanged.Validation on Python 3.13.14 with imports verified from this checkout passed all 573 vLLM tests,
pre-commit run --all-files, the private guardwords scan of this PR's diff and description, andgit diff --check.The author's historical GRPO rollout with Claude Code SWE agents and a Qwen-family policy reported that affected requests were served when enabled; this AI-assisted integration adds no new real-model rollout evidence, and normal GitHub review and current-head CI gates still apply.