Skip to content

feat: dense arch pack for remaining Llama-family architectures (Mistral/Ministral, ExaOne, InternLM, Baichuan, GLM4, and more) #499

Closed

Description

@inureyes

Part of #493

Context

The remaining Llama-family architectures are structurally close to Llama with minor deltas. Target set: Mistral variants / Ministral3, ExaOne / ExaOne4, InternLM2 / InternLM3, Baichuan, Apertus, SolarOpen, GLM4, Ernie4.5 (dense), Hunyuan (dense), Seed-OSS, MiMo, Step3.5.

Once the shared attention core (#494) lands, each is a config flag plus a small forward delta.

Scope

  • Add each architecture via Config::from_json flags and the shared forward.
  • Per-family weight_names ordering in iree.rs.
  • Files: emitter/config.rs, emitter/model.rs, iree.rs.

Acceptance criteria

  • Each listed architecture loads and generates via the CLI MLXCEL_BACKEND=xla path and serves via mlxcel-server.
  • Single-sequence path is token-exact vs an HF fp32 oracle on a real checkpoint.
  • Serve path is reference-exact via xla_batch_bench.

Dependencies

#494 (shared attention core). Uses #496 (validation harness) when available.

Activity

  1. added
    type:enhancementNew features, capabilities, or significant additions
    area:architectureArchitecture and code structure changes
    status:blockedBlocked by dependencies or other issues
    on Jun 30, 2026
  2. added a commit that references this issue on Jul 1, 2026
  3. self-assigned this
    on Jul 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:architectureArchitecture and code structure changespriority:lowLow prioritystatus:doneCompletedtype:enhancementNew features, capabilities, or significant additions

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions