Repository navigation
fix: exclude grammar-only requests from loop detection (#977) - #982
Merged
Merged
Conversation
Schema-valid structured output can legitimately repeat the same value beyond any fixed token-loop threshold, causing valid JSON to be truncated with an ordinary stop reason. Remove grammar constraints from the Gemma 4 automatic activation signal while retaining tool declarations and replayed tool content. Explicit request and global overrides keep their existing precedence, and route-level regression coverage pins both the structured-output fix and tool-call protection. Validation: cargo test --lib server::request_options::tests; cargo check --lib --tests; cargo clippy --lib --tests -- -D warnings. Refs #977
inureyes
added
type:bug
be-free-snowboy
pushed a commit
to freestylejs/snowbrain-mlxcel
that referenced
this pull request
Oct 27, 2026
…ablup#982) ## Summary Prevent Gemma 4 grammar-constrained structured output from being silently truncated when a schema-valid value repeats beyond the token-loop threshold. Automatic detection remains active for tool-shaped prompts, while explicit request and global overrides keep their existing precedence. ## What changed - Remove grammar constraints from the family-default loop-detection amplifier signal across chat, Responses, and completion routes. - Preserve automatic detection for rendered tool declarations and replayed `tool_calls` / `tool_call_id` message content. - Add a regression fixture for the 30-identical-values `json_schema` case and retain route-level tool-protection coverage. - Update supported-model and environment-variable documentation to describe the resulting policy and tradeoff. ## Test plan - [x] `cargo test --lib server::request_options::tests` - [x] `cargo check --lib --tests` - [x] `cargo clippy --lib --tests -- -D warnings` - [x] `cargo fmt --all` - [x] `git diff --check` Closes lablup#977
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prevent Gemma 4 grammar-constrained structured output from being silently truncated when a schema-valid value repeats beyond the token-loop threshold. Automatic detection remains active for tool-shaped prompts, while explicit request and global overrides keep their existing precedence.
What changed
tool_calls/tool_call_idmessage content.json_schemacase and retain route-level tool-protection coverage.Test plan
cargo test --lib server::request_options::testscargo check --lib --testscargo clippy --lib --tests -- -D warningscargo fmt --allgit diff --checkCloses #977