Repository navigation
fix(models): PhiMoE norm epsilon key and SAM antialiased bicubic a #2465
Copy link
Copy link
Closed
Labels
area:modelsModel architectures, weights, loading, metadataModel architectures, weights, loading, metadatapriority:lowLow priorityLow prioritystatus:doneCompletedCompletedtype:bugBug fixes, error corrections, or issue resolutionsBug fixes, error corrections, or issue resolutions
Description
Problem / Background
Two small numeric-parity gaps left over from merged model PRs. Implement both in one PR, run the verification pass once at the end, and keep pushes to a minimum.
- PhiMoE (found in PR fix(models): port upstream numeric fixes for four model families #2457):
phimoe::ModelArgsreadslayer_norm_eps(default 1e-6), but Phi-3.5-MoE checkpoints spell itrms_norm_eps: 1e-5, so the checkpoint value is ignored and the norms run at the wrong epsilon. - SAM vision encoder (left bit-identical in PR fix(vision): port LFM2-VL resampling, Qwen3-VL Conv3d guard and pos_embed #2459): the antialiased bicubic resize uses a = -0.75, while ATen and mlx-vlm use a = -0.5 for the antialiased kernel.
Current Behavior
- Item 1:
src/models/phimoe.rs:67declareslayer_norm_epswithdefault_layer_norm_eps(line 137); it feedsLayerNorm::newat lines 510, 520 and 610. - Item 2:
src/vision/encoders/resample_matrix.rs:70hasconst A: f32 = -0.75forResampleKernel::Cubic(line 42).interp_matrix(in, out, kernel, antialias)also serves non-antialiased callers, where -0.75 is correct. The SAM call site issrc/vision/encoders/deepseekocr_sam.rs:81and:82. Other antialiased callers:kimi_vl_pos_emb.rs:171and:173useResampleKernel::Triangle(not affected);deepseekocr_clip.rs:190is plain cubic without antialias;nemotron_h_nano_omni.rs:56is antialias=False. Re-grepinterp_matrixbefore editing to confirm no other cubic antialiased caller exists.
Proposed Solution
- Read the key mlx-lm
phimoe.pyuses for its norm epsilon and match it. Accept both spellings inModelArgs(serde alias or explicit fallback) with the checkpoint'srms_norm_epswinning when present, keeping 1e-6 only when neither key exists. - Select a = -0.5 only when
kernel == Cubic && antialias && scale > 1(the stretched-support path), keeping -0.75 on the non-antialiased path. Verify against mlx-vlm and torchinterpolate(..., mode="bicubic", antialias=True).
Scope
In scope: src/models/phimoe.rs, src/vision/encoders/resample_matrix.rs, and tests.
Out of scope: other vision encoders' non-antialiased resizes.
Implementation Notes
- Reuse: existing fixtures style in
resample_matrixtests andphimoeconfig tests. - Constraints: non-antialiased cubic output must stay bit-identical.
- Edge cases: upsampling with antialias=True (kernel support is not stretched; confirm which coefficient torch uses and mirror it), scale exactly 1.
- Error handling: none new.
Acceptance Criteria
- Item 1: a config test parsing
{"rms_norm_eps": 1e-5}expects 1e-5 and one without either key expects 1e-6. Fails on current main for the first case. - Item 1: a short greedy or logit-trace check on a local Phi-3.5-MoE checkpoint, compared against mlx-lm, is recorded in the PR.
- Item 2: a resize parity test against a fixture generated by torch
interpolate(..., mode="bicubic", antialias=True)(or mlx-vlm) on a downsampling case, passing within a stated tolerance. Fails on current main. - Item 2: a test showing the non-antialiased cubic matrix is unchanged.
- SAM's real preprocessing path (
deepseekocr_sam.rs) uses the corrected matrix.
Verification
cargo test --profile test-fast phimoe -- --test-threads=1
cargo test --profile test-fast resample_matrix -- --test-threads=1
cargo test --profile test-fast deepseekocr -- --test-threads=1
cargo fmt --all -- --check
cargo clippy --profile test-fast -- -D warnings
Generate the torch fixture offline and commit only the small reference values.
Metadata
Metadata
Assignees
Labels
area:modelsModel architectures, weights, loading, metadataModel architectures, weights, loading, metadatapriority:lowLow priorityLow prioritystatus:doneCompletedCompletedtype:bugBug fixes, error corrections, or issue resolutionsBug fixes, error corrections, or issue resolutions
Activity
fix(models): PhiMoE norm epsilon key and SAM antialiased bicubic a (#…