Skip to content

fix: repair qwen3_coder, Mistral and pythonic tool-call parser gaps - #2487

Merged
adrysn merged 8 commits into
mainfrom
fix/issue-2392-tool-call-parser-gaps
Oct 20, 2026
Merged

adrysn merged 8 commits into
mainfrom
fix/issue-2392-tool-call-parser-gaps

Conversation

@adrysn

@adrysn adrysn commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Summary

Fixes three tool-call parser gaps in src/server/tool_calls/formats.rs and passes a model-emitted tool-call id through to clients.

  • qwen3_coder: function and parameter names follow upstream \s*([^\s<>]+)>?, so a dropped > no longer folds the next tag into the name. A parameter name that cannot be matched is skipped, not the whole call. A name that runs to end of input (truncated generation) is rejected. The quoted-name search stops at the next <, which keeps the scan linear on adversarial input.
  • Mistral [ARGS]: try_mistral_bracket loops over ([\w-]+)\s*\[ARGS\]\s* headers and reads one JSON value after each one, the equivalent of raw_decode. Parallel and hyphenated calls now parse. A truncated trailing call is dropped. Parsing stops at MISTRAL_MAX_CALLS (1024).
  • Tool-call id: ParsedToolCall gains id: Option<String>. try_mistral_nemo sets it from the list element only when it matches ^[A-Za-z0-9_-]{1,64}$ and is unique in the result. Chat (build_tool_call_responses, streaming routes/chat.rs) and Responses (responses_translator.rs, streaming routes/responses.rs) use it as the call id when present and mint one otherwise. The Responses fc_ item id is still minted.
  • Pythonic: a quote-aware scanner that tracks ()[]{} depth replaces the regex. It emits one call per name(...), requires the closing ], and converts JSON-style dict literals to objects. split_pythonic_commas tracks all three bracket kinds. The 64-candidate retry cap counts only real name( candidates, so [1] or markdown links before a call do not use up retries. The "whole body is valid JSON, decline" guard is unchanged.
  • Docs: docs/server-features.md, docs/responses-api.md and a CHANGELOG entry describe the id passthrough.

Not in scope

  • The Anthropic surface (anthropic_translator.rs, routes/anthropic.rs) still mints toolu_ ids and ignores call.id.
  • Python-only dict literals ({'k': True}, None) stay raw strings. Only JSON-syntax dicts become objects.

Test plan

  • MLXCEL_BUILD_METAL=0 cargo test --profile test-fast -p mlxcel tool_calls::: 453 passed, 0 failed (CPU-only MLX build on macOS)
  • MLXCEL_BUILD_METAL=0 cargo clippy -p mlxcel --lib --tests -- -D warnings: clean
  • cargo fmt --check: clean
  • New tests for each acceptance item, plus regression tests: EOF-truncated name, linear quoted-name scan, unterminated pythonic list, non-call brackets before a call, invalid or duplicate Mistral ids, Mistral call cap, and full parse_tool_calls dispatch
  • Real-checkpoint smoke (Qwen3-Coder-30B-A3B, Devstral-Small-2507) not run

Resolves #2392

qwen3_coder took the function and parameter name as everything up to the next '>', so a dropped '>' swallowed the following tag into the name. Names now stop at the first whitespace, '<' or '>' (upstream mlx-lm rule) and an unmatchable parameter name is skipped.

Mistral bracket parsing now loops over `NAME[ARGS]` headers (hyphenated names allowed) and reads one JSON value per header, returning the calls collected before a truncated one. ParsedToolCall gains `id: Option<String>`, set from the Mistral Nemo list element and preferred over a minted id in the Chat (streaming and non-streaming) and Responses (streaming and non-streaming) builders.

Pythonic parsing replaces the lazy regex with a scanner that tracks quotes and ()[]{} depth, so multiple calls, nested calls, and dict/list values survive. Dict literals coerce to JSON objects.

Validated with the touched parser code compiled in an isolated harness (206 formats tests pass); the full crate could not be built here because cmake is missing.

Refs #2392
@adrysn adrysn added status:review Under review type:bug Bug fixes, error corrections, or issue resolutions priority:medium Medium priority area:inference Generation, sampling, decoding (incl. speculative, DRY) modeltype:text Text-only language model labels Oct 20, 2026
With the closing `>` now optional, a truncated opener such as `<function=read_file` at end of input matched as a complete name and produced a spurious second call, failing qwen3_coder_malformed_missing_closing_tags_keeps_prior_calls. A name that reaches end of input with no terminator is now treated as truncated: a function name is rejected and a parameter is skipped. Also collapses a nested `if` in the pythonic dict coercion flagged by clippy.

Tests: tool_calls:: 446 passed; clippy -D warnings on lib and tests clean.

Refs #2392
The quoted-name branch of match_qwen_name searched for `>` to the end of input, so a run of malformed `<parameter="x` openers (now skipped instead of breaking the loop) rescanned the whole tail each time: about 300 ms for 0.5 MB, quadratic in input size. The scan now stops at the next `<` and falls back to the unquoted rule, which also stops a quoted name with a dropped `>` from swallowing the next tag.

The pythonic scanner accepted a call list that never closed with `]`, so prose such as `see [note(x) for details` or a list comprehension produced a call even with no tools to filter it. The list must now close right after its last call, matching the brief and the old `)]` requirement.

Also refresh the stale pythonic and Mistral doc comments, and add a parse_tool_calls dispatch test for the three #2392 inputs. tool_calls tests (450), clippy and fmt pass.

Refs #2392
@adrysn

adrysn commented Oct 20, 2026

Copy link
Copy Markdown
Member Author

Implementation Review Summary

Intent

Fix the qwen3_coder dropped-> name bug, parse parallel and id-bearing Mistral calls, and parse multi-call and nested pythonic lists (#2392).

Findings Addressed

  • Quoted qwen3_coder parameter names with no > rescanned to end of input on every skipped opener, an O(N^2) guard regression (about 300 ms per 0.5 MB). The scan now stops at the next < (HIGH, 76f314c)
  • Pythonic scanner accepted lists that never close with ], so prose like see [note(x) for details became a call (MEDIUM, 76f314c)
  • Stale pythonic and Mistral doc comments; added a parse_tool_calls dispatch test for the three inputs (LOW, 76f314c)

Remaining Items

  • The Anthropic tool_use builder (parsed_call_to_tool_use) still mints toolu_ ids and ignores call.id (MEDIUM). It was outside the brief's three sites and uses its own id convention.
  • Python-only dict literals ({'k': True}) stay raw strings. Only JSON-syntax dicts become objects (LOW)
  • An empty model id ("id":"") is passed through as-is (LOW)
  • The Mistral bracket loop has no max-calls cap like the other parsers. It is linear, so this is defensive only (LOW)

Verification

  • All stated requirements implemented
  • No placeholder/mock code remaining
  • Integrated into project code flow
  • Project conventions followed
  • Existing modules reused where applicable
  • No unintended structural changes
  • Tests pass (tool_calls:: 450, clippy, fmt; the real-checkpoint smoke was not run on this host)

@adrysn

adrysn commented Oct 20, 2026

Copy link
Copy Markdown
Member Author

Security and Performance Review

No CRITICAL or HIGH findings, so no commits. Fuzzing about 300k random inputs (including multibyte characters) through try_mistral_bracket, try_mistral_nemo, try_pythonic, try_qwen3_coder and parse_tool_calls produced no panics. Adversarial probes all ran in linear time: 100k Mistral calls took 17 ms, and 300 KB pythonic worst cases ([a( repeated, an open quote) took about 40 ms.

Remaining Items

  • MEDIUM, scan_pythonic_calls (formats.rs): non-call [ characters such as [1] or markdown links use up the 64-retry budget, so a valid [f(x=1)] after 65 of them returns None. The old regex found it, so this is a regression from main. Fix: count a retry only when the scan reaches name(.
  • MEDIUM, try_mistral_nemo id passthrough: the model id has no length or charset check, and empty or duplicate ids pass through unchanged ("id":"" twice gives [Some(""), Some("")]). This does not harm the server: ids only go into serde-escaped JSON, and server code never uses call_id as a map key. It can break client-side matching of tool results to calls. Fix: keep the id only if it matches ^[A-Za-z0-9_-]{1,64}$ and is unique in the parse result, otherwise use None.
  • MEDIUM (carried over): the Anthropic parsed_call_to_tool_use ignores call.id.
  • LOW, try_mistral_bracket: there is no max-calls cap. The call count is bounded by output length, so this is a consistency fix only. Fix: break at 1024 calls, the same as the sibling parsers.

Verification

  • cargo test --profile test-fast -p mlxcel tool_calls::: 450 passed
  • cargo clippy -p mlxcel --lib --tests -- -D warnings: clean
  • cargo fmt --check: clean

The pythonic scanner spent one of its 64 retries on every `[`, so 65 non-call brackets like `[1]` hid a later valid call that main found. Only candidates that reach `name(` and scan for the closing `)` now count.

try_mistral_nemo now keeps a model-supplied id only when it matches `^[A-Za-z0-9_-]{1,64}$` and is unique within the result; otherwise a fresh id is minted. try_mistral_bracket stops after 1024 calls like the sibling parsers.

Tests: tool_calls:: 453 passed; clippy -D warnings on lib and tests clean; cargo fmt --check clean.

Refs #2392
Chat Completions and Responses now return a tool-call id the model emitted instead of always minting one. Document that in server-features.md and responses-api.md, and add an Unreleased Fixed entry covering the qwen3_coder, Mistral and pythonic parser fixes. No Korean docs tree exists in this repository, so no translation was added.

Validation:
- cargo test --profile test-fast -p mlxcel tool_calls:: (453 passed)
- cargo clippy -p mlxcel --lib --tests -- -D warnings and cargo fmt --check clean

Refs #2392
@adrysn adrysn added status:done Completed and removed status:review Under review labels Oct 20, 2026
@adrysn
adrysn merged commit c7db5a2 into main Oct 20, 2026
27 checks passed
@adrysn
adrysn deleted the fix/issue-2392-tool-call-parser-gaps branch October 20, 2026 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:inference Generation, sampling, decoding (incl. speculative, DRY) modeltype:text Text-only language model priority:medium Medium priority status:done Completed type:bug Bug fixes, error corrections, or issue resolutions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: Repair qwen3_coder, Mistral and pythonic tool-call parser gaps

1 participant