Skip to content

fix: Write the layer_norm VJP bias gradient on ROCm #2632

Closed

Description

@inureyes

Problem

fast::LayerNormVJP::eval_gpu in src/lib/mlxcelverse/overlays/rocm/mlx/backend/rocm/layer_norm.hip:423-429 allocates gb when has_gb is true and leaves // TODO: Implement proper column reduction for gb. Nothing writes it, so the bias gradient of fast::layer_norm on ROCm is uninitialized device memory. Found by code reading; layer_norm_vjp_past_the_row_grid_cap_matches_host in tests/rocm_hip_grid_clamp/row_grid.rs:269 checks only wrt 0 and 1, which is why it passes. Reproduce it first.

Fix

Port the CUDA path (mlx/backend/cuda/layer_norm.cu, LayerNormVJP::eval_gpu): before the VJP kernel launches, col_reduce(encoder, g, gb, Reduce::ReduceType::Sum, {0}, plan) with a ContiguousStridedReduce plan over n_rows x axis_size, then encoder.set_input_array(gb) when g was donated into gx or gw_temp. The reduction must run before the kernel because a donated g is overwritten. gb keeps the dtype MLX gives it (the bias dtype); the expected value is sum(g, all axes but the last), the fallback in mlx/fast.cpp. Reuse the existing col_reduce call used for gw at layer_norm.hip:507-510.

Acceptance Criteria

  • wrt = 2 matches a host reference (column sums of g) for f32, f16 and bf16 with weight and bias; the test fails on current main.
  • The same check at n_rows > 65535, extending layer_norm_vjp_past_the_row_grid_cap_matches_host (the row_grid split of fix(rocm): split per-row softmax and norm grids past the HIP limit #2627); fails on main.
  • Bias without weight: make the weight argument of fast_layer_norm_vjp_for_test (src/lib/mlxcel-core/cpp/mlx_cxx_bridge.cpp:4244, src/lib/mlxcel-core/src/lib.rs:1530) a nullable *const MlxArray as fast_scaled_dot_product_attention does for mask, and test gb with no weight; fails on main.
  • New LOCAL_FIXES.md item (next number after 63) naming layer_norm.hip; no CORE_RESIDUAL.diff change unless a core file changes.

Verification

Implement everything in one PR with one verification pass at the end and as few pushes as possible.

scripts/rocm_gpu_guard.sh --hold -- cargo test --profile test-fast --features rocm --test rocm_hip_grid_clamp -- --test-threads=1
scripts/rocm_gpu_guard.sh --hold -- make verify-rocm

Activity

  1. added a commit that references this issue on Oct 26, 2026
    0ee8f03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    platform:linuxLinux (CUDA / packaging) specificpriority:highHigh prioritystatus:doneCompletedtype:bugBug fixes, error corrections, or issue resolutions

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions