Repository navigation
fix: InvalidConfig labels non-LLaVA loader errors as invalid LLaVA config #986
Description
Problem / Background
HostPreprocessorError::InvalidConfig in src/multimodal/host_preprocessor.rs:1074 is declared as:
#[error("invalid LLaVA host-preprocessor config: {0}")]
InvalidConfig(String),Several of the messages constructed with that variant have nothing to do with a LLaVA checkpoint (line numbers against main at c3e134ba5):
host_preprocessor.rs:95, inXlaVisionBackendPolicy::from_value:MLXCEL_XLA_VISION_BACKEND must be auto, host, or iree; got {other:?}. This is family independent, and on the Qwen2-VL path it fires fromfrom_env()after the family has been identified as Qwen2-VL, never LLaVA.host_preprocessor.rs:158:Qwen2-VL XLA vision has no MLX fallback; MLXCEL_XLA_VISION_BACKEND=host is unsupported.host_preprocessor.rs:178:Qwen2-VL XLA image execution requires the xla-iree feature(the#[cfg(not(feature = "xla-iree"))]arm).
Two related cases found while checking the above:
host_preprocessor.rs:150:failed to identify model family from {path}: {error}. This one fires before any family is known, so claiming LLaVA is guaranteed wrong.host_preprocessor.rs:237:MLXCEL_XLA_VISION_BACKEND=iree requires the xla-iree feature. This is reachable only underload_llava_image_preprocessor, so the LLaVA prefix is not false, but the message is about an env var and a build feature rather than a config field, and the combined string reads oddly.
Impact is diagnostic only, but it is user facing. Both surfaces that load the preprocessor stringify the error:
src/backend/xla.rs:111wraps it insidecreate_sessionasOpenXLA image preprocessor load failed: {error}.src/server/batch/xla_preprocess.rs:74maps the loader error witherror.to_string()and passes it through the worker ready channel, which fails server startup with that text.
So for a Qwen2-VL checkpoint an operator sees:
OpenXLA image preprocessor load failed: invalid LLaVA host-preprocessor config: Qwen2-VL XLA image execution requires the xla-iree feature
They are told a LLaVA config is invalid while loading a Qwen2-VL checkpoint, which sends them looking in the wrong place.
Proposed Solution
Either of:
- Make the variant text family neutral, for example
invalid OpenXLA host-preprocessor config: {0}, or - Split a family-specific variant off so LLaVA weight and config failures keep the LLaVA wording while the policy, family-detection, and Qwen2-VL arms get their own variant.
Prefer whichever keeps the LLaVA weight-loading and config messages as informative as they are today. WeightLoad (failed to load LLaVA host-preprocessor weights: {0}) is only produced from the LLaVA loader paths in src/loading/vlm_llava.rs, so it can stay as is unless the split makes a matching rename natural.
Acceptance Criteria
- The variant text no longer claims LLaVA for family-independent messages (
MLXCEL_XLA_VISION_BACKENDpolicy parsing, model-family detection failure) or for Qwen2-VL messages. - The actionable content of each message is preserved verbatim: the env var name
MLXCEL_XLA_VISION_BACKENDand its accepted values, and the--features xla-ireerebuild instruction. - LLaVA-specific config and weight failures still name LLaVA, so their diagnostics do not regress.
-
cargo test --release --lib --features metal,accelerate multimodal::host_preprocessorstays green. - Any assertion that matches on the message text in
src/multimodal/host_preprocessor_tests.rsor thesrc/server/batch/xla_preprocess.rstests is updated. Current state: the host-preprocessor tests match on the variant withmatches!(host_preprocessor_tests.rs:300and:314) rather than on the display string, and the batch tests assert on other substrings, so no text assertion is expected to break. Re-check after the change.
Technical Considerations
This was split out of #966 to keep that PR surgical. #966 deliberately left the wrapper text alone and only touched the Qwen2-VL loader contract; it does extend the message at host_preprocessor.rs:178 with the rebuild instruction (; rebuild mlxcel with --features xla-iree (this family has no MLX vision fallback)), so whoever picks this up should rebase on the merged #966 and preserve whatever final text landed there.
No behavior change is intended: the same conditions must still fail, with the same variant discriminant unless a new variant is introduced, and only the rendered prefix changes.
Activity
tomatotomata commentedon Aug 2, 2026
I was thinking of taking this diagnostics fix. I would introduce the smallest family-neutral or split error variant needed so policy, family-detection, and Qwen2-VL failures no longer claim LLaVA, while preserving the existing LLaVA-specific wording. I would add focused assertions for the rendered messages, re-check the affected tests, and run cargo fmt plus the scoped host-preprocessor test command from the issue. Does that scope fit, and may I take it?
adrysn commentedon Sep 2, 2026
Closing: the wording this issue asks for is already on
main.Commit 52b7a5d (#916) changed
HostPreprocessorError::InvalidConfigto render asinvalid host-preprocessor config: {0}andWeightLoadasfailed to load host-preprocessor weights: {0}, both family neutral, and added aFamilyMismatchvariant (incompatible multimodal family: expected LLaVA, got {actual}) that names LLaVA only where the family is actually checked. As a result the policy-parsing message (MLXCEL_XLA_VISION_BACKEND must be auto, host, or iree; got ...), the family-detection failure (failed to identify model family from ...), and the Qwen2-VL messages no longer claim a LLaVA config, while the env var values and the--features xla-ireerebuild instruction are preserved verbatim.Verified against
mainat 3eeb196:grep -rn 'invalid LLaVA host-preprocessor config' src/returns no hits. The residual item (keeping LLaVA-specific wording on the LLaVA-only config and weight paths) was resolved in the family-neutral direction that this issue listed as its first option, so no further change is planned.