Skip to content

fix(models): close the GELU formula-rule gaps left after #2785 - #2804

Merged
inureyes merged 1 commit into
mainfrom
fix/issue-2794-gelu-rule-gaps
Oct 31, 2026
Merged

inureyes merged 1 commit into
mainfrom
fix/issue-2794-gelu-rule-gaps

Conversation

@inureyes

@inureyes inureyes commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Summary

Implements #2794: five GELU formula-rule gaps outside #2548's row list, following the section-0b rule (the checkpoint config decides; an omitted key falls back to the Hugging Face config-class default; mlxcel follows HF where mlx-lm or mlx-vlm departs).

What changed

  • GPT-BigCode reads activation_function (absent = the GPTBigCodeConfig default gelu_pytorch_tanh), resolves it once at load onto MLP, and rejects non-GELU names in validate()
  • Idefics2 and Granite 4 Vision pass use_fast_gelu = false; the tower config decides (Idefics2 resolves its omitted key to the Idefics2VisionConfig default via default_missing_hidden_act_to_siglip)
  • Shared VisionHiddenActivation maps every ACT2FN GELU name through GeluActivation::from_hf_name (gelu_new / gelu_fast now tanh) and rejects relu / silu as load errors; Bunny's sparse-config fallback propagates instead of swallowing
  • A declared gelu_pytorch_tanh on the shared SigLIP MLP runs the compiled kernel like the omitted-key path (VisionMlpActivation::PytorchTanh removed, SiglipDefaultTanh folded in)
  • Dead C++ compiled_gelu removed with its three FFI tests and both compile-audit entries; the XLA llava vision config accepts the same GELU name set
  • docs/gelu-call-sites.md: rows T5, S6, S7, a new S-cfg row, the shared-resolution paragraph, the inventory rows, and the compiled_gelu row updated; GeluActivation "Used by" doc comment extended

Test plan

  • cargo fmt --all -- --check passes
  • cargo clippy --profile test-fast --features cuda -p mlxcel -p mlxcel-core -p mlxcel-xla --all-targets -- -D warnings passes (exit 0)
  • cargo check (test-fast, cuda, the three packages) clean in 11m37s
  • cargo test -p mlxcel-xla vision_config: 10 passed, 0 failed
  • GPU lib tests (gpt_bigcode, siglip, vision::config, idefics2, granite4, llava, core gelu) with --test-threads=1 and MLX_ENABLE_TF32=0 under gpu-lock (run after the push, when the lock freed): 111 passed, 0 failed, 1 ignored on CUDA (GB10)
  • VLM smoke (greedy mlxcel generate --image + one mlxcel-server image request each on models/idefics2-8b-4bit, models/granite-4.0-3b-vision-4bit, models/smolvlm-instruct-bf16, MLX_ENABLE_TF32=0, run after the push, on tests/fixtures/test_image.png): all six coherent. generate: idefics2 "A bright orange colored fabric with no patterns or text."; granite4 "The image provided is a solid block of a uniform color, specifically a bright orange or reddish-orange hue."; smolvlm "A solid orange background with no other objects, text, or scenery.". server: idefics2 "The image is a bright orange background with no other objects in the image."; granite4 "The image is a solid, uniform block of a bright orange color."; smolvlm "A solid orange background with no other objects, text, or elements present."
  • GPT-BigCode smoke: not run, no checkpoint exists on this host (as the issue notes); the Metal benches use models/mlx/gpt_bigcode-santacoder on a Mac host

Closes #2794

Five rule-breakers outside #2548's row list, all following the section-0b rule: the checkpoint config decides, an omitted key falls back to the Hugging Face config-class default, and mlxcel follows HF where mlx-lm or mlx-vlm departs.

- GPT-BigCode now reads activation_function (absent or null means the GPTBigCodeConfig default gelu_pytorch_tanh), resolves it once at load onto MLP, and rejects non-GELU names in validate(); santacoder's declared tanh runs the compiled f32 kernel instead of the erf mlx-lm applies.
- Idefics2 and Granite 4 Vision stop forcing use_fast_gelu (mlx-vlm's FastGELUMLP): the tower config decides, with Idefics2 resolving an omitted key to the Idefics2VisionConfig default via default_missing_hidden_act_to_siglip.
- The shared VisionHiddenActivation maps every ACT2FN GELU name through GeluActivation::from_hf_name (gelu_new / gelu_fast now tanh) and rejects non-GELU names as load errors instead of silently running erf; Bunny's sparse-config fallback propagates the error rather than swallowing it.
- A declared gelu_pytorch_tanh on the shared SigLIP MLP runs the compiled kernel like the omitted-key path (VisionMlpActivation::PytorchTanh removed, SiglipDefaultTanh folded in).
- The dead C++ compiled_gelu is removed with its three FFI tests and both compile-audit entries; the XLA llava vision config accepts the same GELU name set. Docs rows T5, S6, S7 and the shared-config rule updated.

Validated: cargo fmt, clippy --all-targets -D warnings (mlxcel, mlxcel-core, mlxcel-xla), the mlxcel-xla vision_config tests (10 passed), and a clean cargo check on the three packages. The GPU lib-test gate and VLM smoke runs were not run before this commit: the GB10 gpu-lock was held by another session; the orchestrator's contract gate covers them. No GPT-BigCode checkpoint exists on this host, so that smoke cannot run here.

Closes #2794.
@inureyes inureyes added the status:review Under review label Oct 31, 2026
@inureyes
inureyes marked this pull request as ready for review October 31, 2026 19:00
@inureyes
inureyes merged commit f87f0da into main Oct 31, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status:review Under review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(models): close the GELU formula-rule gaps left after #2785

1 participant