feat(inference): implement or explicitly reject control-vector options #1449

Parent

• epic(server): close llama-server b10621 compatibility gaps #1431

Problem

b10621 supports multiple control vectors, per-vector scales, and an inclusive layer range. mlxcel has no demonstrated equivalent, and these flags change model activations rather than merely configuring loading, so accepting them as compatibility no-ops would be incorrect.

Scope

• Classify --control-vector, --control-vector-scaled, and --control-vector-layer-range as an implemented MLX feature or explicit unsupported behavior.
• If implemented, define the supported artifact format, tensor naming, dtype conversion, device placement, scale composition, and layer-index mapping for mlxcel model families.
• Support multiple vectors and deterministic scale/order semantics matching b10621.
• Validate dimensions and model identity before serving; do not partially apply an invalid vector set.
• Expose the resolved vector configuration in redacted diagnostics and model identity/cache keys.

Acceptance criteria

• Unsupported use fails at startup and no control-vector flag remains accepted and ignored: --control-vector and --control-vector-scaled parse (hidden) and any configured vector fails startup before the model load with a diagnostic naming logit-level steering (--lang-bias / per-request logit_bias) as the nearest mlxcel feature; --control-vector-layer-range alone is accepted as inert because without vectors it configures nothing, exactly as upstream. Classification chosen: explicit unsupported behavior (the issue's own first branch), not an implementation, because per-layer activation additions would require forward-pass hooks across every model family and a partial implementation would be exactly the half-finished breadth the project forbids.
• Implemented vectors validate shape, layer range, finite scale, and checkpoint compatibility atomically. Not applicable: no implemented path exists; a scale of zero is still a configured vector set and is rejected as a whole rather than partially applied (tested), so nothing can be applied non-atomically.
• Multiple/scaled vectors and inclusive layer boundaries have deterministic unit tests: the rejection of both vector flags (including scale zero) and the inertness of the two-value inclusive range are pinned in src/cli/ggml_compat_args_tests.rs.
• Prompt/KV caches cannot be reused across different control-vector configurations: only one configuration (none) can ever serve, because every vector set fails startup, so cross-configuration reuse is impossible by construction.

Numerical validation

For an implemented path, record a teacher-forced baseline, enabled vector trace, repeatability, and decided-position disagreement; verify scale zero reproduces the baseline within the documented tolerance.

Numerical validation note: the requested teacher-forced baseline, enabled-vector trace, and scale-zero baseline reproduction apply, per this issue's own wording, to an implemented path. The chosen classification is explicit rejection, so no sampling or forward arithmetic changed and the trace comparison is identically zero by construction; the scale-zero case is instead pinned as a startup rejection test (a zero-scaled vector is still a configured vector set, not a silent no-op).