fix: Structured output returns an empty-mask error on a long uniform array that the schema permits #978

Problem / Background

A structured-output request whose schema is trivially satisfiable fails with a 500-class error instead of returning a completion. The error text blames the caller's schema, but the schema in question imposes no length bound at all.

Measured reproduction

Verified locally against main at 053443d4, model models/gemma-4-12b-it-4bit, CUDA build of mlxcel-server, plain POST /v1/chat/completions (no streaming).

Request parameters:

• response_format: {"type":"json_schema","json_schema":{"name":"vals","schema":{"type":"object","properties":{"values":{"type":"array","items":{"type":"integer"}}},"required":["values"],"additionalProperties":false}}}
• temperature: 0, seed: 42, max_tokens: 400
• Prompt: Return the values array containing the number 7 repeated 40 times.

Response body:

{"error":{"message":"Generation error: structured output: constrained-decoding error: structured-output matcher returned an empty mask: no token can extend the partial output without violating the schema. The model is stuck - this is usually a sign that the schema is too restrictive for the supplied prompt.","type":"server_error","code":null}}

Zero loop-detection cuts were logged for this request, so this is not the loop detector firing. The same schema with "repeated 30 times" did not reach this path; it was truncated by loop detection instead, which is filed separately.

Why the error message is misleading

{"type":"array","items":{"type":"integer"}} has no maxItems and permits an arbitrarily long run of integers. For any partial output consistent with that schema there is always a legal continuation: at minimum a , followed by another digit, or a closing ]. An empty mask should therefore be unreachable for this schema. Either the matcher is losing state over a long repetitive sequence, or the mask is being computed against a stale or truncated matcher position. Whatever the cause, the user-facing message attributes an engine-side defect to the caller's schema.

Impact

• A well formed request with a satisfiable schema returns a server error instead of a completion.
• Generation work already performed is discarded when the sequence aborts.
• The remedy suggested by the message ("your schema is too restrictive") sends the caller down a path that cannot fix the problem.

Relevant code

Line numbers on main at 053443d4:

• src/server/structured.rs:747 emits the message.
• apply_structured_mask_to_logits (src/server/structured.rs:722) builds the bias and is where the empty-mask condition is detected. It counts allowed bits with allowed.iter().take(vocab_size).filter(|x| **x).count(), where vocab_size is the model's vocab_size_hint, and errors when that count is zero.
• StructuredOutputConstraint::compute_mask (src/server/structured.rs:345) returns an empty slice when self.matcher.is_stopped(), before any mask is computed.
• Call sites in src/server/batch/scheduler.rs: prefill at :5491, decode at :6440 and :6811, all routed through the apply_structured_mask helper at :2382.

Two engine-side paths worth checking, either of which would produce this message without the schema being at fault:

1. Vocab-size reconciliation in apply_structured_mask_to_logits. The allowed-bit count only considers positions below vocab_size_hint, and positions in [matcher_vocab, vocab_size_hint) are conservatively masked out. It is worth confirming that this reconciliation cannot zero out an otherwise non-empty mask, and that the matcher vocabulary and the logits vocabulary agree for this model.
2. Stopped-matcher handling. No call site checks StructuredOutputConstraint::is_stopped() before applying the mask, and compute_mask returns an empty slice in exactly that state. If the matcher stops while the scheduler still runs another decode step for the sequence, the next mask application would raise this error even though the constrained output was already complete.

Suggested investigation steps

1. Reproduce with the request above and capture the partial output at the exact step where the mask goes empty.
2. Determine whether the matcher's internal position matches the tokens actually committed, especially across a long run of identical tokens.
3. Check whether the failure depends on array length, on the specific integer value, or on a token-boundary effect where a multi-digit number or a repeated separator token straddles a schema element boundary.
4. Confirm whether the matcher had already stopped at the failing step, which would make this a stop-handling bug rather than a constraint violation.
5. If the matcher is behaving correctly and the constraint genuinely admits no continuation, correct the message so it reports the engine state accurately instead of blaming the caller's schema.

Acceptance Criteria

• The reproduction above returns a schema-conforming completion instead of a server error.
• Root cause identified as matcher state loss, vocab-size reconciliation, or stopped-matcher handling, and fixed at that layer.
• A regression test covers a long uniform array (40 or more identical integers) under an unbounded items: integer array schema, exercising the same constrained-decoding path the server uses.
• The empty-mask error message no longer asserts that the caller's schema is too restrictive unless that has actually been established.
• The fix is verified through the real /v1/chat/completions request path, not only at the unit level.

Technical Considerations

Found while verifying PR #969, but unrelated to it and present before that change. #969 touches loop-detection calibration and the Gemma 4 default-on gate; this failure occurs on the constrained-decoding path with no loop-detection cuts logged.