Disable SSE4.2 in noavx native builds

[modelpath-dev]

Summary

• The published noavx CPU backend still compiles with -msse4.2. ggml turns GGML_SSE42 on for a normal (non-cross) compile even when GGML_NATIVE=OFF.
• That is an illegal instruction on CPUs that stop at SSSE3, which is the crash in Getting a Illegal instruction (core dumped) when trying to run #1422. lscpu there has no avx, sse4_1, or sse4_2.
• noavx jobs on Linux, musl, Windows, and the macOS x64 Rosetta build now pass -DGGML_SSE42=OFF. gcc/clang jobs also pin -march=x86-64 so auto-vectorization cannot put SSE4 back in. MSVC only gets the CMake flag.

Test plan

• Configure current llama.cpp with the old noavx flags. CMake prints Adding CPU backend variant ggml-cpu: -msse4.2 GGML_SSE42.
• Configure with the new flags. CMake prints an empty CPU variant and GGML_SSE42=OFF.
• objdump of libggml-cpu.so: old build contains pblendvb / pinsrd (SSE4.1). New build does not.
• After the next Update Binaries run and a new LLamaSharp.Backend.Cpu publish, Getting a Illegal instruction (core dumped) when trying to run #1422 should load without SIGILL. Until then the FAQ notes the self-compile path.

[martindevans]

Thanks for putting this together, it all looks good to me.

Could you please trigger a test action run on your fork (I can't do it in this repo until it's merged).

To do that:

1. Go to "Actions" tab in your fork
2. Select "Update Binaries" in the list on the left
3. Click "Run Workflow" (top right of the action panel)
4. Link the action run here

[modelpath-dev]

Triggered Update Binaries on the fork from this branch (fix/1422-noavx-sse42, llama.cpp master):

https://github.com/modelpath-dev/LLamaSharp/actions/runs/34667855880

[modelpath-dev]

Update Binaries completed successfully on the fork from this branch:

https://github.com/modelpath-dev/LLamaSharp/actions/runs/34669867568

llama.cpp d3146f2 (master at dispatch). Every compile job and Gather Binaries passed, including Linux/Windows/musl noavx with SSE4.2 off.

The first run failed on Windows ARM64 only. ClangCL + CMake PCH disagrees on C17 for ggml-cpu, so llama.dll was never produced. That is unrelated to the noavx flags. I pinned that job to windows-2022 and set CMAKE_DISABLE_PRECOMPILE_HEADERS=ON so the ARM64 artifact builds.

[martindevans]

Thanks very much for working on this! Ready to merge once the CI passes :)

[modelpath-dev]

@martindevans

Hi Martin, thanks again for the thorough review on this one. Walking through the noavx native builds and the fork CI runs taught me a lot about how the binary pipeline fits together. I would like to keep contributing to LLamaSharp, especially around native builds and packaging.

I am currently looking for a role, so if there is ever build or maintenance work you would like a hand with, I would be glad to help. Either way, I will keep an eye on the open issues.