Skip to content

fix(xla): update Gemma 3n emitter to the shared-layer query offset #2466

Closed

Description

@inureyes

Problem / Background

PR #2457 fixed the Gemma 3n shared-KV layer query offset in the MLX path (src/models/gemma3n.rs, the offset comes from rope_offset at line 439, matching mlx-lm's cache[0].offset). The mlxcel-xla emitter was deliberately left mirroring the old offset, so MLX-vs-XLA Gemma 3n parity now diverges. Related to epic #566.

Current Behavior

src/lib/mlxcel-xla/src/emitter/gemma3n_emit_ops.rs:790-800 builds query_positions for shared-KV layers (lw.wk.is_none()) as positions + real_len, with a comment saying it reuses the concrete layer's advanced cache offset. That is the pre-#2457 behavior. Separately, the xla-diagnostics call sites in src/models/gemma3n.rs were updated by hand and never compiled, because the xla feature does not build on the GB10 host.

Proposed Solution

Update the emitter so shared-layer query RoPE positions equal what the fixed MLX path computes. Read the post-#2457 gemma3n.rs logic first and port exactly that, then fix the comment. Compile the diagnostics call sites on a host that builds the xla feature and fix any errors.

Scope

In scope: gemma3n_emit_ops.rs, the xla-diagnostics call sites in src/models/gemma3n.rs, the Gemma 3n parity test.

Out of scope: the rest of epic #566.

Implementation Notes

  • Reuse: the existing MLX-vs-XLA Gemma 3n parity check.
  • Constraints: needs a host where --features xla builds (not GB10). If none is available, say so in the PR and ask the maintainer to run it.
  • Edge cases: prefill versus decode steps, sliding versus full layers.
  • Error handling: none new.

Acceptance Criteria

  • The emitter computes shared-layer query positions as the fixed MLX path does.
  • The existing MLX-vs-XLA Gemma 3n parity check passes on a host that builds xla, and failed there before the change.
  • The xla-diagnostics call sites in gemma3n.rs compile with --features xla.
  • The XLA Gemma 3n serve path uses the corrected emitter.

Verification

cargo build --features xla
cargo test --features xla gemma3n -- --test-threads=1
cargo fmt --all -- --check
cargo clippy --features xla -- -D warnings

A pass is a clean build with the diagnostics sites compiled and the parity check green.

Activity

  1. added
    type:bugBug fixes, error corrections, or issue resolutions
    area:modelsModel architectures, weights, loading, metadata
    and removed on Oct 20, 2026
  2. inureyes commented on Oct 21, 2026

    @inureyes
    MemberAuthor

    Closing as completed by #2513. One correction to this issue's verification section: the root crate has no plain xla feature, so cargo build --features xla cannot exist; the real surface is xla-iree (production) and xla-diagnostics (cuda + iree + diagnostics), which is what PR #2513 used. The prefill divergence the issue targets is resolved (token-prefill logits max_abs 1.9375 -> 0.8203, within the 1.5 envelope, matching top1 on both sides); the residual prefix-decode drift predates #2457 and is recorded, not gating, per the #932 contract. The previously-unbuilt xla-diagnostics call sites in src/models/gemma3n.rs compile on the real surface.

  3. added a commit that references this issue on Oct 21, 2026
    d013373
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:modelsModel architectures, weights, loading, metadatapriority:lowLow prioritystatus:doneCompletedtype:bugBug fixes, error corrections, or issue resolutions

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions