[inureyes]

Summary

• load the qualified LLaVA host preprocessor alongside XLA sessions and report multimodal capability only when the complete path is available
• route CLI images through the shared bounded decode and prepared-prefill contract
• add a bounded asynchronous image-preprocessing stage to continuous-batch serving, preserving logical usage counts separately from effective prefill lengths
• recognize nested LLaVA text configs and language_model.* checkpoint weights required by the real LLaVA Interleave target
• keep audio, video, logprobs, and structured output explicitly unsupported, with per-request image failure and cancellation isolation

Validation

• cargo fmt --all -- --check
• cargo clippy -p mlxcel --lib --tests --features xla-iree -- -D warnings
• cargo check -p mlxcel --lib --tests --features xla-backend
• cargo check -p mlxcel --lib --tests --features xla-iree
• focused XLA preprocessing/worker tests: 9 passed
• nested LLaVA config regression: 1 passed
• LLaVA Interleave 384/14 loader regression: 1 passed
• XLA scaffold/capability regression: 1 passed
• real IREE CLI image generation with llava-interleave-qwen-0.5b-bf16: generated one token successfully
• real OpenAI-compatible streaming image request: returned The, finish_reason=length, usage 24/1/25, [DONE]; subsequent /health returned status=ok

The full 4,583-test xla-backend library run progressed through the affected tests, including the updated XLA scaffold regression, but the process aborted at the existing CUDA-only models::bitnet::tests::bitlinear_matmul_known_ternary_case because this host has no CUDA backend.

Closes #861

[inureyes]

Rebased this PR onto e739ca1 (#894, multimodal RoPE position state). The updated head is 18d6d0d.

There was one content conflict in src/lib/mlxcel-xla/src/emitter/config.rs, limited to adjacent test blocks. The resolution keeps both contracts:

• #894: explicit PreparedPositions, position_mode, signed rope_delta, Qwen2/Qwen3 M-RoPE layouts, and fail-closed handling when rope_type: "mrope" lacks mrope_section.
• #861: LLaVA/LLaVA-Next wrapper parsing through nested text_config, including inherited wrapper quantization.

I also audited the automatically merged iree.rs path. The #894 prepared-position validation and 1D/M-RoPE dispatch remain unchanged; this PR only adds the complete language_model.* dense-checkpoint namespace selection needed by the LLaVA target. LLaVA host preprocessing now uses the rebased PreparedPositions::Sequential contract.

[inureyes]

Post-rebase validation evidence:

• cargo test -p mlxcel-xla --lib: 195 passed, 6 ignored. This includes the nested LLaVA config regression, M-RoPE fail-closed/config tests, 1D prepared-position canonicalization, signed-delta tests, and mixed text/M-RoPE slot reuse.
• Focused XLA image preprocessing and worker tests: 9 passed. This includes bounded admission, cancellation/capacity failures, malformed-image isolation beside active text, logical-vs-effective lengths, unsupported options/modalities, and the pending-preprocess poll timeout regression.
• LLaVA Interleave 384/14 loader regression: 1 passed.
• Text-only XLA scaffold/capability regression: 1 passed.
• cargo check -p mlxcel --lib --tests --features xla-backend: passed.
• Pinned-IREE cargo check -p mlxcel --lib --tests --features xla-iree: passed.
• Pinned-IREE cargo clippy -p mlxcel --lib --tests --features xla-iree -- -D warnings: passed.
• Root xla-backend library suite, serial, with only the known CUDA-only BitNet test excluded: 4,469 passed, 113 ignored, 0 failed, 1 filtered out. The excluded test is models::bitnet::tests::bitlinear_matmul_known_ternary_case; it aborts on this CPU-only host with [cuda_kernel] No CUDA back-end.
• Real IREE CLI image request with llava-interleave-qwen-0.5b-bf16: generated one token successfully (<|im_end|>, 154.31 s).
• Real OpenAI-compatible streaming image request: returned The, finish_reason: "length", usage 24/1/25, and [DONE]. A subsequent /health returned status: "ok", active_sequences: 0, and queue_depth: 0.
• cargo fmt --all -- --check and git diff --check: passed.
• CSV diff: none.

[inureyes]

Implemented the review hardening in 8880d6e.

Correctness changes

• Preserve declared and resolved image/audio/video counts from PreparedChatRequest through every HTTP route and ModelRequest.
• Keep the generic MLX resolver tolerant, while making XLA fail closed when any declared image is dropped or only partially resolved/decoded.
• Reject declared audio/video from declaration metadata, so malformed payloads cannot disappear into a text-only request.
• Apply the same declared-media guard to the disaggregated text-only router.
• Reject --output-audio before the XLA CLI early return.
• Qualify XLA multimodal capabilities and dispatch with the same runtime + host-preprocessor predicate.

Regression coverage

• Deterministic blocking preprocessor: in-flight cancellation, FIFO progress, bounded queue-full behavior, and result-receiver disconnect.
• Active text beside failed/cancelled image work, followed by successful slot reuse.
• Logical prompt usage vs expanded effective-prefill observability.
• Text/LLaVA/missing-artifact/unsupported-VLM capability and startup matrix.
• All-invalid, partially invalid, oversized, and partially decoded image requests.
• Declared audio/video and unsupported structured/logprobs admission.

Validation

• Root XLA+IREE library suite (serial, known CUDA-only BitNet test excluded): 4,488 passed, 113 ignored, 0 failed, 1 filtered
• mlxcel-xla library suite: 195 passed, 6 ignored
• Default and XLA+IREE cargo clippy --lib --bins --tests -- -D warnings: passed
• XLA+IREE cargo check, test compilation, cargo fmt --all -- --check, and git diff --check: passed
• Real IREE CLI image request: exit 0, generated <|im_end|> in 151.91 s
• Real OpenAI-compatible streaming image request: returned The, finish_reason=length, usage 24/1/25, then [DONE]
• Post-request /health: status=ok, active_sequences=0, queue_depth=0
• Real XLA CLI --output-audio: rejected with the expected unsupported-backend error
• CSV diff: none

The repository-wide --all-targets Clippy invocation still reports only the pre-existing unrelated clippy::print_literal in examples/qmm_gemv_microbench.rs:250; the changed library, binaries, and tests pass with warnings denied.

[inureyes]

Addressed the final CLI cardinality review finding in c18d5b6.

• decode_xla_cli_images() still reads each payload under the shared configured limits and calls the shared decoder exactly once.
• Immediately after that decode, the CLI now requires decoded.len() == paths.len() and rejects partial image execution with the declared and decoded counts.
• The regression sends one valid fixture plus one malformed payload through the real shared tolerant decoder, verifies that only the valid image survives there, and verifies that the new CLI boundary fails closed.

Validation:

• default targeted CLI regression: 1 passed
• XLA+IREE CLI regressions (partial multi-image decode and --output-audio): 2 passed
• default and XLA+IREE cargo check --bin mlxcel: passed
• default and XLA+IREE test-aware cargo clippy --bin mlxcel --tests -- -D warnings: passed
• cargo fmt --all -- --check and git diff --check: passed
• CSV diff: none

The prior head 8880d6e completed the real IREE CLI image E2E (exit 0, <|im_end|>) and OpenAI streaming server E2E (The, usage 24/1/25, [DONE], then healthy empty slots). This follow-up changes only the CLI's post-decode cardinality guard and its unit test; the all-valid payload path still returns the same decoded images to the same prepared-prefill call, and no server/runtime code changed, so the five-minute real E2E was not repeated.