kleidiai : add SME2 f32 kernel

[chaxu01]

Overview

This patch integrates the KleidiAI SME2 F32 GEMM/GEMV kernels into the kleidiai backend and enable runtime dispatch to the optimized SME2 implementations when supported by the target system.

Additional information

Benchmark results on an Apple M4 Pro (2 SME units) using Llama-3.2-1B F32 show substantial improvements over the native F32 implementation:

Threads	Native F32 (t/s)	SME2 F32 (t/s)	Speedup
1	19.70	457.57	23.2×
2	37.78	892.35	23.6×
4	77.28	561.52	7.3×
6	104.57	452.55	4.3×
8	122.40	609.17	5.0×

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: <!-- mention: NO

[chaxu01]

@ggerganov — just wanted to highlight this PR when you have a chance. The patch integrates the KleidiAI SME2 F32 GEMM/GEMV kernels into the KleidiAI backend and adds runtime dispatch to the optimized SME2 implementations when supported by the target system.

Happy to provide any additional details if helpful. Thanks!

[chaxu01]

@ggerganov, I've just updated this PR with an improvement to the SME2 F32 path by enabling dynamic scheduling.

The original implementation used static column partitioning, which works well up to the number of available SME units but doesn't scale well beyond that. With dynamic scheduling enabled, the F32 kernels maintain much better utilization when the thread count exceeds the available SME units.

Threads	Static SME2 F32	Dynamic Scheduling
1	457.57	450.83
2	892.35	1067.71
4	561.52	938.98
6	452.55	1179.90
8	609.17	1114.89

Whenever you have a chance, I'd appreciate it if you could take a look. Happy to address any feedback or make further changes if needed. Thanks!

[chaxu01]

@ggerganov — just wanted to follow up on this PR when you have a chance.

The PR has been updated with dynamic scheduling for the SME2 F32 path, which significantly improves scaling once the thread count exceeds the available SME units.

When you have a chance to take a look, I'd really appreciate any feedback or suggestions you may have. I'm happy to address any comments or make further improvements if needed. Thanks!

[ggerganov]

Thanks, I missed the notifications for this.