Enable MLAS f16<->f32 cast kernel on Apple ARM64

[Justin Chu (justinchuby)]

What this enables

The NEON f16↔f32 conversion kernel (cast_kernel_neon.cpp) is excluded on all Apple targets today — mlas.h gates MLAS_F16VEC_INTRINSICS_SUPPORTED behind #if !defined(__APPLE__), and cmake/onnxruntime_mlas.cmake excludes the source with if (NOT APPLE). The comment marks this "temporary"; it was never revisited. macOS ARM64 therefore falls back to a scalar Source[i].ToFloat() loop.

This enables only that cast kernel, on macOS arm64 only, via a new MLAS_CAST_F16_NEON_SUPPORTED macro kept separate from MLAS_F16VEC_INTRINSICS_SUPPORTED, so fp16 arithmetic, i8mm and bf16 stay disabled on Apple.

Scope

Gated on TARGET_OS_OSX plus MLAS_TARGET_ARM64 in C++, and CMAKE_SYSTEM_NAME STREQUAL "Darwin" plus arm64 in CMake — deliberately not CMake's APPLE, which is also true for iOS, tvOS and visionOS. No universal2, no x86_64 Apple slices, no iOS/tvOS/visionOS.

No -march flag is needed

vcvt_f32_f16 / vcvt_f16_f32 are baseline AArch64 conversion instructions (FCVTL/FCVTN) — arm_neon.h guards them under #if (__ARM_FP & 2), not __ARM_FEATURE_FP16_VECTOR_ARITHMETIC. FEAT_FP16 governs fp16 arithmetic, which this kernel does not perform. An earlier revision added -march=armv8.2-a+fp16; it was unnecessary and has been removed.

NaN contract

For NaN results the tests assert NaN-ness (exponent all ones, mantissa non-zero) and the sign bit — the payload is deliberately not asserted. Raw-bit equality is retained for every non-NaN value.

That asymmetry is required, not a convenience: hardware FCVTN preserves the NaN payload, while the software reference canonicalises every NaN to 0x7E00. Measured by cross-compiling and executing real AArch64 NEON:

input	hardware FCVTN	software reference
f32 sNaN 0x7FA00000	0x7F00	0x7E00
f32 qNaN 0x7FC12300	0x7E09	0x7E00

A bit-exact NaN assertion would fail on any real AArch64 target.

Test coverage

Test	What it establishes
CastFp16ShortExecuteTest (lengths 1–259)	Bit-exact conversion for non-NaN values, including lengths that are not multiples of the vector width
CastFp16SpecialValuesTest	Normals, denormals, ±0, ±Inf, qNaN, sNaN, and a true round-to-nearest-even tie at 1 + 2^-11
CastFp16KernelDispatchTest	GetMlasPlatform().CastF16ToF32Kernel is non-null — i.e. the NEON kernel is dispatched rather than the scalar fallback

Where these have and have not run

Emulated AArch64. I cross-compiled the kernel with aarch64-linux-gnu-g++ -O2 -static and ran it under qemu-aarch64-static: all special values and all 17 bulk lengths pass. This validates the NEON instruction semantics — which is what the NaN behaviour above turns on — but it is emulation on a Linux host, not Apple Silicon, and I am not presenting it as hardware validation.

Native Apple Silicon. The cpu / build-and-test (arm64, arm64, …) lanes in mac.yml run with machine == target, which satisfies the workflow's test gate, and build.py --test invokes onnxruntime_mlas_test directly on non-Windows hosts. Those jobs pass on this PR, including the "Running Tests (build.py --test)" step.

To be precise about the limits of that: I have confirmed the test step executes and succeeds on native arm64 runners, and that the binary it runs contains these tests. I have not been able to retrieve per-test output from the job log to show CastFp16KernelDispatchTest by name. If a maintainer would like that made explicit, a targeted test registration or a filtered invocation could be added — I did not want to change what the MLAS suite runs across all platforms just to surface one line.

No performance claims

Nothing was measured. This replaces a scalar conversion loop with a vectorised one on macOS arm64; whether and where that matters is not established here.

CI status

All checks are green: 86 passing, 1 skipped (CodeQL, expected for this diff). Two transient infra issues surfaced during review and are documented here for the record — neither was caused by this change:

• 4. Build Minimal (Globally Allowed Types) (Linux CPU Minimal Build E2E) sat queued for ~20.5 hours while its 9 sibling jobs in the same run completed normally within minutes — an orphaned runner-scheduling issue. Cancelling and re-triggering (rerun-failed-jobs) picked up a healthy runner slot; the job then passed immediately.
• React Native CI iOS E2E Tests showed cancelled, not a real test failure: the Detox/iOS-simulator step hung after boot and sat idle until the job's timeout-minutes: 90 fired (react_native.yml). This diff touches only 6 MLAS C++/CMake files and nothing under js/react_native; 2 of the last 5 main-branch runs of this same workflow show the identical hang, confirming a pre-existing, recurring flake. Rerunning passed cleanly.

No code changes were needed for either issue.

[Copilot]

Pull request overview

Enables NEON FP16↔FP32 conversion kernels on macOS ARM64 while keeping other FP16 arithmetic disabled.

Changes:

• Adds a macOS ARM64-specific capability macro and CMake source inclusion.
• Enables NEON cast-kernel dispatch.
• Expands conversion, special-value, and dispatch tests.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
cmake/onnxruntime_mlas.cmake	Includes the NEON cast kernel for Darwin ARM64 builds.
onnxruntime/core/mlas/inc/mlas.h	Defines macOS ARM64 cast support.
onnxruntime/core/mlas/lib/cast_kernel_neon.cpp	Updates kernel support documentation.
onnxruntime/core/mlas/lib/mlasi.h	Exposes NEON cast declarations under the new capability.
onnxruntime/core/mlas/lib/platform.cpp	Enables runtime dispatch under the new capability.
onnxruntime/test/mlas/unittest/test_cast_fp16.cpp	Expands conversion and dispatch tests.

Suppressed comments (1)

onnxruntime/test/mlas/unittest/test_cast_fp16.cpp:146

• This F32→F16 case set has only positive NaNs and no inputs whose half result is denormal. Therefore it does not exercise the NaN sign contract or the advertised denormal coverage in this conversion direction. Add negative NaNs and exact positive/negative half-denormal equivalents.

    std::vector<float> f32_input = {
        0.0f, -0.0f,
        std::numeric_limits<float>::infinity(),
        -std::numeric_limits<float>::infinity(),
        std::numeric_limits<float>::quiet_NaN(),
        std::numeric_limits<float>::signaling_NaN(),
        1.00048828125f};

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Hariharan Seshadri (hariharans29)]

Review — PR #31993: Enable MLAS f16↔f32 cast kernel on Apple macOS arm64

Summary of changes (6 files, targeted +100/−~10)

• mlas.h: adds a new MLAS_CAST_F16_NEON_SUPPORTED macro, gated on __APPLE__ && MLAS_TARGET_ARM64 && TARGET_OS_OSX (via <TargetConditionals.h>). Deliberately kept separate from the existing MLAS_F16VEC_INTRINSICS_SUPPORTED so fp16 arithmetic, i8mm, and bf16 stay disabled on Apple — this only opens the conversion path.
• cast_kernel_neon.cpp: header comment updated to reflect the two-macro condition. Kernel body untouched.
• mlasi.h and platform.cpp: declaration and dispatch-table assignment guards widened from MLAS_F16VEC_INTRINSICS_SUPPORTED && MLAS_TARGET_ARM64 to (MLAS_F16VEC_INTRINSICS_SUPPORTED || MLAS_CAST_F16_NEON_SUPPORTED) && MLAS_TARGET_ARM64 — symmetric.
• cmake/onnxruntime_mlas.cmake: adds cast_kernel_neon.cpp to mlas_platform_srcs on CMAKE_SYSTEM_NAME STREQUAL "Darwin" (deliberately not the CMake APPLE which is true on iOS/tvOS/visionOS as well). No compile flag added.
• test_cast_fp16.cpp: adds TestSpecialValues (IEEE 754 corner cases + RNE tie) and TestKernelIsDispatched (dispatch-table pointer non-null), plus more length values in the bulk test.

The technical claim is correct — I verified it

vcvt_f32_f16 and vcvt_f16_f32 are baseline AArch64 (FCVTL/FCVTN in the F16C-in-AArch64 sense — arm_neon.h guards them under #if (__ARM_FP & 2), not __ARM_FEATURE_FP16_VECTOR_ARITHMETIC). FEAT_FP16 governs float16x8_t arithmetic (fadd/fmla on packed halves), which this kernel does not perform. The kernel body (cast_kernel_neon.cpp lines 47–177) uses only baseline conversions plus load/store/reinterpret intrinsics — no packed fp16 arithmetic anywhere. The prior "hardware specific compilation flag" comment in mlas.h was correct in aggregate for the fp16 kernel family, but overbroad for the cast path specifically. This PR punches a precise, minimal hole in that gate.

Scope isolation is well-done

I traced every other consumer of MLAS_F16VEC_INTRINSICS_SUPPORTED:

• activate_fp16.cpp, dwconv.cpp, pooling_fp16.cpp, halfgemm.cpp, halfgemm.h (declaration and MlasHalfGemmGetDispatch), hgemm_kernel_neon.cpp, gelu_neon_fp16.cpp, erf_neon_fp16.cpp, eltwise_kernel_neon.cpp, softmax_kernel_neon.cpp, rotary_embedding_kernel_neon.cpp, qnbitgemm_kernel_neon.cpp, qnbitgemm_kernel_neon.h, fp16_common.h — all use float16x8_t arithmetic and are correctly not widened by this PR. On macOS arm64, MlasHalfGemmGetDispatch() still returns &MlasHalfGemmDispatchDefault (scalar), which is what you want. ✓
• The platform.cpp block that assigns ErfFP16KernelRoutine / GeluFP16KernelRoutine / TanhFP16KernelRoutine is gated on MLAS_F16VEC_INTRINSICS_SUPPORTED (not the widened guard), so those stay nullptr on Apple. ✓

Only the two cast-dispatch slots (CastF16ToF32Kernel, CastF32ToF16Kernel) get wired up, which is exactly the stated scope.

NaN test contract is right

I initially raised an eyebrow at the asymmetric NaN assertion (std::isnan(dispatch) + sign only, vs. bit-exact for non-NaN). The description walks through why: hardware FCVTN preserves the payload bits (truncated to fit in 10 mantissa bits) while the software reference MLAS_Float2Half canonicalizes every NaN to 0x7E00. The two examples in the description (0x7FA00000 → 0x7F00 hardware vs. 0x7E00 software) match FCVTN's spec (it copies bits [22:13] into the fp16 mantissa, which for 0x7FA00000 = binary(exp=all-ones, mant=010 0000 0000 0000 0000 0000) gives fp16 mantissa 1 0000 0000 0 = 0x300, so the result is 0x7C00 | 0x300 = 0x7F00). ✓ Bit-exact NaN assertion here would break the test on every real AArch64 target, so this contract is required, not stylistic.

Universal2 clarification worth making

The PR description says "No universal2", but I traced the CMake and that's misleading — the description means "we don't add any x86_64-Apple dispatch", not "universal2 arm64 slices don't get the kernel". Concretely:

• When OSX_ARCHITECTURES = "arm64;x86_64", ONNXRUNTIME_MLAS_MULTI_ARCH = TRUE (onnxruntime_mlas.cmake line 486).
• The ARM64 branch (line ~641) runs and appends cast_kernel_neon.cpp to mlas_platform_srcs.
• onnxruntime_add_static_library(onnxruntime_mlas_arm64 ${mlas_platform_srcs}) builds those sources with OSX_ARCHITECTURES "arm64", so __aarch64__ is defined, MLAS_TARGET_ARM64 is set, TARGET_OS_OSX is 1 → MLAS_CAST_F16_NEON_SUPPORTED is defined for the arm64 slice.
• The x86_64 slice's compilation of platform.cpp runs with MLAS_TARGET_ARM64 undefined, so the whole widened guard is dead code there. ✓

That's actually the correct behavior (arm64 slice gets NEON, x86_64 slice stays scalar). The PR description should be tweaked to say "no new x86_64-Apple dispatch; universal2 arm64 slice inherits the macOS-arm64 behavior naturally" so the Copilot review comment (medium severity) doesn't get read as "code is wrong". Either fix the description or add a one-line CMake comment explaining the interaction with ONNXRUNTIME_MLAS_MULTI_ARCH. I'd take the latter — it's the more discoverable place.

Comments / suggestions

1. Stale comment above MLAS_F16VEC_INTRINSICS_SUPPORTED in mlas.h. Lines 92–95 still say:

   // Had to temporary disable fp16 under APPLE ARM64, as compiling
   // the source files require a hardware specific compilation flag.
   // When building an universial binary for APPLE, this flag would
   // cause trouble for x64 target.
   

   This is now imprecise — the "compiling requires a hardware specific compilation flag" claim is not true for cast_kernel_neon.cpp specifically (per the whole point of this PR). Tighten to something like "fp16 arithmetic kernels need -march=armv8.2-a+fp16 and are disabled on Apple; the f16↔f32 cast kernel is baseline AArch64 and is enabled separately below via MLAS_CAST_F16_NEON_SUPPORTED." Prevents the next contributor from staring at the same "temporary" hedge for another five years.

2. Copilot low-priority Set up CI with Azure Pipelines #1 (negative NaN coverage) — take it. It's four lines, and it converts the sign-bit assertion from vacuous to load-bearing. Trivially:

   const uint16_t kNegQNaN = 0xFE00;
   const uint16_t kNegSNaN = 0xFC01;

   and add to special_bits. On the F32→F16 direction, similarly add -std::numeric_limits<float>::quiet_NaN() / -std::numeric_limits<float>::signaling_NaN(). Right now if a kernel silently zeroed the sign bit on negative NaNs, this test would not catch it.

3. Copilot low-priority Remove vsts test runner in cmake file #2 (test-length range wording) — the description says "lengths 1–259" but the test iterates 18 discrete values in {1, 2, 3, 5, 7, 9, 15, 16, 17, 31, 32, 63, 64, 128, 255, 256, 1024, 65536}. The test is fine as a boundary set; just fix the description to "18 boundary-focused lengths from 1 to 65 536, including all non-multiples of the 4/8-lane vector widths." Precision matters here since the CI story rests on this test running natively.

4. Consistency nit in the widened guard. The condition (defined(MLAS_F16VEC_INTRINSICS_SUPPORTED) || defined(MLAS_CAST_F16_NEON_SUPPORTED)) && defined(MLAS_TARGET_ARM64) appears in two places (mlasi.h, platform.cpp). Given it will likely repeat if a follow-up enables another Apple-arm64 kernel, consider centralizing:

   // in mlas.h or a shared internal header
   #if defined(MLAS_F16VEC_INTRINSICS_SUPPORTED) || defined(MLAS_CAST_F16_NEON_SUPPORTED)
   #define MLAS_HAS_NEON_F16_CAST
   #endif

   then both guards become #if defined(MLAS_HAS_NEON_F16_CAST) && defined(MLAS_TARGET_ARM64). Not a blocker.

5. TestKernelIsDispatched compiles to nothing on x86 CI hosts. That's intentional and documented in the comment (This test compiles to nothing on platforms without a vectorised kernel). But RegisterTests() unconditionally returns 1 and registers the test, meaning on x86 you'll see a CastFp16/KernelDispatched gtest listing that runs an empty body. Fine, but if you want the test to be visible only on platforms it can meaningfully assert on, wrap the RegisterTest call itself in the same #if. Cosmetic.

Non-issues I re-verified

• <TargetConditionals.h> include in a public MLAS header is safe: the include is nested inside #if defined(__APPLE__), and on Apple SDKs the header is always available (part of the base SDK, not a specific framework). No -framework dependency added. ✓
• Kernel body uses float16_t via float16x4_t / vreinterpret_*_f16. That type is baseline on AArch64 (as opposed to _Float16 arithmetic, which is FEAT_FP16-gated). Compiles clean on any AArch64 target with arm_neon.h. ✓
• No regressions to non-Apple ARM64 paths: on Linux arm64 (or Windows ARM64), MLAS_F16VEC_INTRINSICS_SUPPORTED is still set the same way, so the widened guard's LHS is true, and the RHS MLAS_CAST_F16_NEON_SUPPORTED is undefined but harmless. Existing behavior unchanged. ✓
• The #include "core/mlas/lib/mlasi.h" in test_cast_fp16.cpp is needed to reach GetMlasPlatform(). Existing tests in that directory already reach into internal headers (platform.cpp grep shows several such patterns). Consistent. ✓
• CI: description accurately describes the two transient infra flakes (Build Minimal (Globally Allowed Types) queue starvation, iOS Detox hang). Both are documented pre-existing issues and are unrelated to this diff. ✓

Recommendation

Approve after (1) the mlas.h "temporary" comment is tightened and (2) the negative-NaN coverage is added. Both are small, don't warrant another round. (3) is a description edit. (4) and (5) are follow-up-worthy nits.

This is a clean, well-scoped, correctly-reasoned enabling PR — the author has already done the hard part of separating conversion (baseline) from arithmetic (FEAT_FP16), and the test contract for NaN handling is exactly right for hardware-vs-software round-tripping.