Skip to content

fix: stop swallowing template rejections - #1511

Merged
inureyes merged 2 commits into
mainfrom
fix/issue-1176-template-rejections
Aug 30, 2026
Merged

inureyes merged 2 commits into
mainfrom
fix/issue-1176-template-rejections

Conversation

@inureyes

@inureyes inureyes commented Aug 30, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Preserve template-rejection sentinel errors through chat request preparation so downstream routes can distinguish deliberate template refusals from engine-side render failures.
  • Return HTTP 400 invalid_request_error from the disaggregated router when a chat template rejects caller-supplied values.
  • Make offline CLI prompt resolution return a named error for template rejections while retaining bare-prompt fallback for non-rejection render failures.
  • Add committed English and Korean technical reports for this open PR because TECHNICAL_REPORTS/.keep-reports is present.

Validation

  • PASS: cargo test --bin mlxcel resolve_cli_prompt
  • PASS: cargo test --bin mlxcel apply_user_chat_template_wraps_prompt_as_user_message
  • PASS: cargo test --bin mlxcel vlm_chat_template
  • PASS: cargo test --lib responses_reasoning_effort_template_rejection_survives_translation
  • PASS: cargo test --lib responses_template_engine_failure_still_uses_prompt_fallback
  • PASS: cargo test --lib router_chat_maps_template_rejection_to_bad_request
  • PASS: cargo test --lib template_rejection
  • PASS: cargo clippy --lib --tests -- -D warnings
  • PASS: cargo clippy --bin mlxcel -- -D warnings
  • PASS: hosted checks on report commit 998a70b: cargo-clippy, cargo-deny, cargo-fmt, OpenXLA feature compile, crate versions, cross-repo refs, kernel dtype keys, llama-compat manifest, Detect changes, and license/cla. MLX pin extraction and OpenXLA feature link skipped by change detection.
  • SKIP: broad cargo workspace tests and cold release builds per watchdog guard.

Reports

  • TECHNICAL_REPORTS/1511-template-rejection-surfaces-20260830.en.md
  • TECHNICAL_REPORTS/1511-template-rejection-surfaces-20260830.ko.md

Closes #1176

Preserve the template rejection sentinel when chat request preparation wraps a render error so disaggregated router endpoints can keep the same 400 invalid_request_error behavior as the single-node routes.

Make offline CLI prompt resolution fallible only for template rejections while retaining bare-prompt fallback for engine-side render failures.

Refs #1176
@inureyes inureyes added status:review Under review type:bug Bug fixes, error corrections, or issue resolutions priority:medium Medium priority area:docs User and developer documentation area:inference Generation, sampling, decoding (incl. speculative, DRY) area:cli Command-line interface / CLI flags status:done Completed and removed status:review Under review labels Aug 30, 2026
Add bilingual technical reports for the template rejection surface fixes in PR #1511.

Refs #1176
@inureyes
inureyes merged commit 9b60258 into main Aug 30, 2026
12 checks passed
@inureyes inureyes self-assigned this Aug 31, 2026
@inureyes
inureyes deleted the fix/issue-1176-template-rejections branch October 6, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:cli Command-line interface / CLI flags area:docs User and developer documentation area:inference Generation, sampling, decoding (incl. speculative, DRY) priority:medium Medium priority status:done Completed type:bug Bug fixes, error corrections, or issue resolutions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(server): /v1/responses and the offline CLI still swallow template rejections that /v1/chat/completions now returns 400 for

1 participant