[inureyes]

Closes #2443

transformers 5 saves rope_theta, rope_scaling and partial_rotary_factor only under a nested rope_parameters block. Only Qwen3 and GraniteMoeHybrid lifted it (#2439). Every other struct that reads the flat keys fell back to its serde default base without an error: a transformers-5 copy of Llama 3.2 1B or Qwen3-30B-A3B on main produces garbage on a 20 KB prompt (below).

Approach

I looked for a shared parse helper that every site already calls and found none: about 125 family load functions read config.json directly, and VLMs nest the block under text_config. So the type is the choke point, as the issue proposes.

• lift_rope_parameters / deserialize_lifting_rope_parameters move to mlxcel_core::rope_parameters (re-exported from models::rope_utils), plus an impl_rope_lifting_deserialize!(Type) macro. Each uncovered struct gets #[serde(remote = "Self")] and one macro line: 86 structs, including the VLM text configs and the three Qwen3-Omni speech configs. Pipeline and tensor-parallel sites parse these same types, so they are covered. No struct gains a field.
• Lifter rules: a flat non-null key wins. It now also lifts partial_rotary_factor and reads the per-layer-type forms, meaning a block whose values are all objects: for Gemma 3 / Gemma 3n / OLMo 3, full_attention gives the base and scaling and sliding_attention gives rope_local_base_freq; for DeepSeek-V4, main gives the base and compress gives compress_rope_theta and the YaRN rope_scaling. It builds rope_scaling from a non-default scheme (Llama 3 llama3 included) or from MRoPE extras like mrope_section. The scheme goes under type only (read type first, then rope_type), because the deepseek_v32 / deepseek_v4 / nemotron alias = "type", alias = "rope_type" fields reject a block that spells both. exaone4 was the one reader that checked only rope_type, and it now accepts type too.
• Call sites outside serde: DFlashConfig::from_json, the Idefics2 Mistral defaults (which used to shadow the nested base), and both vlm_qwen.rs ad hoc lifts now use the shared lifter. JinaVlmTextConfig::from_json falls back to the nested base, and the Phi-4 text-field copy now carries rope_parameters.
• EXAONE-MoE and Mistral 4 used to let the nested base win. Following the issue, a flat key now wins there too. exaone4's rope_scaling is now #[serde(default)], because a default-scheme transformers 5 save has none.

Validation

• rope_parameters_lift_tests::every_flat_rope_theta_config_lifts_rope_parameters_and_flat_wins is table-driven over all 89 parse paths (the 86 lifted structs, Mistral 4, and the DFlash and Jina from_json paths). It checks that nested-only gives 123456 and flat 500000 plus nested gives 500000. Required non-RoPE fields are filled from serde's own errors, never rope_theta / rope_scaling. More tests: Llama 3 nested llama3 resolves RopeScalingKind::Llama3, Gemma 3 per-layer-type, Phi partial_rotary_factor, and DeepSeek-V4 compress YaRN (added in review; not revert-run). With the fix stashed and only the tests kept, the first four all fail, and every one of the 89 rows reports a failure. The lifter unit tests are in mlxcel-core.
• Real checkpoints: transformers-5 copies of ~/models_archive/mlx/llama-3.2-1b-instruct-4bit and ~/models_archive/mlx/qwen3-30b-a3b-4bit, with only config.json rewritten (flat keys moved into a rope_parameters block identical to the one transformers 5.20.1 emits for that config) and the weights symlinked. Command: MLXCEL_SDPA_DETERMINISTIC=1 gpu-lock run -- mlxcel generate -m <dir> -p "<first 20000 bytes of CONTRIBUTING.md + a question>" -n 64 --temp 0 --show-reasoning. On the branch, both copies match their originals byte for byte, and the originals match main. On main (self-parity checked), the Llama copy degenerates into and, and, ... and the Qwen3 copy into . The. The. ....
• fmt, clippy (--lib --tests -D warnings for mlxcel and mlxcel-core), the affected-family test filters (647 tests) and the repo contract checks pass locally.

Flat-wins follows the issue. transformers 5 itself gives the nested value priority, but the two only disagree when a config carries both with different values, and none of the reference checkpoints does.

Changes during review

The review found no CRITICAL or HIGH issues. Two follow-ups were applied: the DeepSeek-V4 main / compress layout, which the first version turned into an all-None rope_scaling, and a "type": null key that hid the rope_type scheme.

Not covered: vision-encoder configs (Pixtral, Llama 4 vision, etc.) and mlxcel-xla, which the issue does not list. DBRX is lifted at attn_config, where transformers 5 still keeps rope_theta, but its top-level rope_parameters is not copied there.