[daemyung-lablup]

Summary

--lora-fuse and mlxcel generate --adapter stored every fused projection of a bf16/fp16 checkpoint as f32. fuse_lora_weights_into added the f32 delta from compute_lora_delta straight onto the base weight, and MLX promotes bf16 + f32 to f32. The fused weights took twice the bytes and every activation downstream of them became f32. The mode that is meant to remove the adapter's decode cost ended up slower than the unfused path.

This PR casts the delta to the base weight's dtype before the add. mlx-lm's LoRALinear.fuse (delta = (...).astype(weight.dtype)) and mlxcel's own embedding fusion (src/lora/embedding.rs) already do this.

Related issues

No issue filed. Found while measuring LoRA serving. Follow-up to #2446, which made the delta f32 explicitly but left the add uncast.

Type of change

• fix — bug fix
• perf — performance improvement (before/after numbers below)

Measurements

Qwen/Qwen3-1.7B (bf16) with a rank-8 adapter trained by mlx_lm lora (16 layers x 7 projections, f32 adapter tensors) on an Apple M4 Pro 48 GB. Decode in tok/s; peak footprint from /usr/bin/time -l.

Path	Before (e160079)	After	Base model, no adapter
mlxcel generate --adapter decode	27.0	64.4 / 61.0	62.7 / 64.1
mlxcel generate --adapter peak footprint	8.6 GB	5.0 GB	5.5 GB
mlxcel-server --lora --lora-fuse decode (/completion timings, 256 tokens)	26–28	60.1–60.5	59.7–60.6

For comparison, the unfused --lora path on the same server runs at 54–55 tok/s.

Parity with mlx-lm

• Method: the same adapter was fused with mlx_lm fuse (mlx-lm 0.31.3). Each fused weight was then compared with W + (a @ b).T * scale cast to bf16, which is what this PR computes.
• Weights: of 805,306,368 elements, 1,976 differ (0.0002%), each by one bf16 ulp (max abs 4.9e-4). The only difference is operation order: mlx-lm scales lora_b before the matmul, and mlxcel scales after.
• Greedy outputs: across 4 prompts × 100 tokens, --adapter and the mlx-lm-fused checkpoint gave the same text on 3. The 4th diverged at an early token, and both continuations stayed in the adapter's style.

Before this PR the fused weights were not bf16 at all.

Test plan

• cargo fmt --all -- --check
• cargo clippy -p mlxcel --lib --tests --features metal,accelerate -- -D warnings
• cargo test --profile test-fast --features metal,accelerate -p mlxcel --lib lora:: -- --test-threads=1: 71 passed, including the new test
• New test a_fused_weight_keeps_the_base_dtype, run against bf16, fp16 and f32 bases. It fails on e160079 (left: 10 (f32), right: 12 (bf16)) and passes with the fix.
• cargo test --workspace --profile test-fast --features metal,accelerate --no-fail-fast -- --test-threads=1: all pass except 6 models::switch_layers::sorted_gather_{layout,history}_tests cases. Those fail identically on main (e160079) without this change. On this M4 Pro each one reports off the f64 reference by 1.017e-2 relative (bf16 ulp is 3.906e-3) from assert_matches_reference in src/models/switch_layers_sorted_gather_test_support.rs. They are unrelated to LoRA, and I have not root-caused them.
• Validated with a real checkpoint: Qwen/Qwen3-1.7B bf16 plus an mlx-lm LoRA adapter (numbers above)

[inureyes]

Hardening review for PR lablup/mlxcel#2527 (follow-up to lablup/mlxcel#2446).

• Purpose & correctness: Casts the f32 LoRA delta returned by compute_lora_delta to mlxcel_core::array_dtype(base_weight) before mlxcel_core::add in src/lora/loader.rs::fuse_lora_weights_into, matching mlx_lm's LoRALinear.fuse and src/lora/embedding.rs::fuse_embedding_pair. Existing shape, Conv1D layout, and quantized-base guards in fuse_lora_weights_into remain intact before the cast and add.
• Security & performance: Prevents unintended bf16/fp16 to f32 promotion on fused projection weights and downstream activations, halving fused weight bytes and restoring bandwidth-bound decode speed (Qwen/Qwen3-1.7B bf16 with rank-8 adapter: 27.0 -> 64.4 tok/s, peak footprint 8.6 -> 5.0 GB). No security issues found.
• Validation: cargo test --profile test-fast -p mlxcel --lib lora:: -- --test-threads=1 (71 passed, 0 failed), verified a_fused_weight_keeps_the_base_dtype fails at HEAD~1 (left: 10, right: 12) and passes with the fix across BFLOAT16, FLOAT16, and FLOAT32, and all 18 code/lint/link CI checks passed (WebUI installed artifact Activity perf gate refused to start due to external 116.8 runnable-task load on the shared GB10 runner).