[inureyes]

Cause

The issue's hypothesis (a float32 q/k/v) was wrong. Logging the SDPA call on the crashing layer showed q bf16 [1,16,1980,256], k/v fp16 [1,8,1980,256], mask f32 [1980,1980]. TurboQuant caches always return fp16 K/V, so a bf16 model hands SDPA a bf16 query over fp16 K/V. SDPA attends in result_type(q, k, v), which is float32 for bf16 + fp16. The bridge cast the additive mask to the query's dtype (bf16), and the CUDA overlay's patched promotion table keeps bf16 + float32 at bf16, so the mask no longer promoted into the float32 output and SDPA threw. Below the window Gemma 4 attends maskless, which is why only prompts past 1024 tokens crashed. Metal's unpatched table promotes bf16 + float32 to float32, so it was CUDA-only.

Fix and why at this boundary

The bridge (fast_scaled_dot_product_attention, the sinks variant, the Metal 4 bridge) is the one place that already adapts mask dtypes to MLX's contract, and its stated purpose was to satisfy "must promote to output type". It picked the wrong target. A shared sdpa_mask_for_inputs now casts floating masks to result_type(q, k, v). The sinks bridge passes attention sinks through the same helper, because SDPA applies the same promotion rule to them (review finding; GPT-OSS greedy output on gpt-oss-20b-mxfp4-q4 is byte-identical before and after). When q/k/v share a dtype this is the same cast as before. mask_in_score_dtype on the chunked path now casts to the query dtype only when K and V share it.

Rejected: casting K/V back to bf16 in Gemma 4 (fixes one model, adds a per-layer K/V copy, and changes turbo4 numerics everywhere), and making turbo caches return the input dtype (changes the cache's fp16 contract that fused turbo kernels rely on).

Validation

• New tests layers::tests::additive_mask_with_bf16_query_over_fp16_kv_is_accepted and gemma4_unified_mask_tests::turbo4_past_window_sliding_mask_is_accepted_by_sdpa (Turbo4Asym RotatingKVCache with bf16 K/V, past-window create_sliding_window_prefill_mask, trim_mask_to_keys, attention_from_ptr, compared with float32 attention). With the bridge change reverted both abort with Mask type must promote to output type float32. With it, they pass.
• layers::tests (100) and models::gemma4 (120) pass. Clippy (-p mlxcel-core --lib --tests, root --lib --tests) and fmt are clean, and so are the scripts/ci checks.
• GB10, ~/models/mlx/gemma-4-26b-a4b-it-4bit, mlxcel-server, 1987-token prompt, 300 tokens at temperature 0, MLXCEL_SDPA_DETERMINISTIC=1. On main, --kv-cache-mode fp16+turbo4 aborts. With the fix it completes, both with no drafter and with --model-draft ~/models/mlx/gemma-4-26b-a4b-it-assistant-bf16 --draft-kind mtp (MLXCEL_MTP_ALLOW_INEXACT=1). The turbo4 output is coherent and matches the FP16 KV arm for the first 287 characters, then diverges on wording, as expected from 4-bit V quantization.
• Follow-ups from review, not in this PR: the CUDA chunk gate (cuda_sdpa_materializes_scores, materializing_sdpa_query_chunk) judges flash eligibility and score bytes from q's dtype, which understates mixed bf16/fp16 attention that runs in float32. The softcap composite still casts its mask to q's dtype and returns v's dtype. The CUDA dtype.cpp overlay header says bf16 + fp16 gives bf16, while its table gives float32.
• A CHANGELOG.md entry is added under Fixed.
• Acceptance criterion 4 (fix(cache): bound the turbo4 sliding append to visible_len #1793) was re-checked with a reverted-fix(cache): bound the turbo4 sliding append to visible_len #2330 build. Results are on fix(cache): bound the turbo4 sliding append to visible_len #1793.

Closes #2336