Skip to content

fix(models): PhiMoE norm epsilon key and SAM antialiased bicubic a #2465

Closed

Description

@inureyes

Problem / Background

Two small numeric-parity gaps left over from merged model PRs. Implement both in one PR, run the verification pass once at the end, and keep pushes to a minimum.

  1. PhiMoE (found in PR fix(models): port upstream numeric fixes for four model families #2457): phimoe::ModelArgs reads layer_norm_eps (default 1e-6), but Phi-3.5-MoE checkpoints spell it rms_norm_eps: 1e-5, so the checkpoint value is ignored and the norms run at the wrong epsilon.
  2. SAM vision encoder (left bit-identical in PR fix(vision): port LFM2-VL resampling, Qwen3-VL Conv3d guard and pos_embed #2459): the antialiased bicubic resize uses a = -0.75, while ATen and mlx-vlm use a = -0.5 for the antialiased kernel.

Current Behavior

  • Item 1: src/models/phimoe.rs:67 declares layer_norm_eps with default_layer_norm_eps (line 137); it feeds LayerNorm::new at lines 510, 520 and 610.
  • Item 2: src/vision/encoders/resample_matrix.rs:70 has const A: f32 = -0.75 for ResampleKernel::Cubic (line 42). interp_matrix(in, out, kernel, antialias) also serves non-antialiased callers, where -0.75 is correct. The SAM call site is src/vision/encoders/deepseekocr_sam.rs:81 and :82. Other antialiased callers: kimi_vl_pos_emb.rs:171 and :173 use ResampleKernel::Triangle (not affected); deepseekocr_clip.rs:190 is plain cubic without antialias; nemotron_h_nano_omni.rs:56 is antialias=False. Re-grep interp_matrix before editing to confirm no other cubic antialiased caller exists.

Proposed Solution

  1. Read the key mlx-lm phimoe.py uses for its norm epsilon and match it. Accept both spellings in ModelArgs (serde alias or explicit fallback) with the checkpoint's rms_norm_eps winning when present, keeping 1e-6 only when neither key exists.
  2. Select a = -0.5 only when kernel == Cubic && antialias && scale > 1 (the stretched-support path), keeping -0.75 on the non-antialiased path. Verify against mlx-vlm and torch interpolate(..., mode="bicubic", antialias=True).

Scope

In scope: src/models/phimoe.rs, src/vision/encoders/resample_matrix.rs, and tests.

Out of scope: other vision encoders' non-antialiased resizes.

Implementation Notes

  • Reuse: existing fixtures style in resample_matrix tests and phimoe config tests.
  • Constraints: non-antialiased cubic output must stay bit-identical.
  • Edge cases: upsampling with antialias=True (kernel support is not stretched; confirm which coefficient torch uses and mirror it), scale exactly 1.
  • Error handling: none new.

Acceptance Criteria

  • Item 1: a config test parsing {"rms_norm_eps": 1e-5} expects 1e-5 and one without either key expects 1e-6. Fails on current main for the first case.
  • Item 1: a short greedy or logit-trace check on a local Phi-3.5-MoE checkpoint, compared against mlx-lm, is recorded in the PR.
  • Item 2: a resize parity test against a fixture generated by torch interpolate(..., mode="bicubic", antialias=True) (or mlx-vlm) on a downsampling case, passing within a stated tolerance. Fails on current main.
  • Item 2: a test showing the non-antialiased cubic matrix is unchanged.
  • SAM's real preprocessing path (deepseekocr_sam.rs) uses the corrected matrix.

Verification

cargo test --profile test-fast phimoe -- --test-threads=1
cargo test --profile test-fast resample_matrix -- --test-threads=1
cargo test --profile test-fast deepseekocr -- --test-threads=1
cargo fmt --all -- --check
cargo clippy --profile test-fast -- -D warnings

Generate the torch fixture offline and commit only the small reference values.

Activity

  1. added
    type:bugBug fixes, error corrections, or issue resolutions
    area:modelsModel architectures, weights, loading, metadata
    and removed on Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:modelsModel architectures, weights, loading, metadatapriority:lowLow prioritystatus:doneCompletedtype:bugBug fixes, error corrections, or issue resolutions

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions