[BensonChen2000]

Overview

Add support to non contiguous unary operation in CUDA backend which was previously unsupported.

Additional information

Tested with:

./build/bin/test-backend-ops.exe perf -b CUDA0 -o ABS,SGN,NEG,STEP,TANH,ELU,RELU,SIGMOID,GELU,GELU_QUICK,SILU,HARDSWISH,HARDSIGMOID,EXP,EXPM1,SOFTPLUS,GELU_ERF,FLOOR,CEIL,ROUND,TRUNC,RELU_SQR

Relate to #14909

Requirements

• I have read and agree with the contributing guidelines: YES

• AI usage disclosure: Yes, AI is used to help me understand the existing code and generate new code.

[saady789]

Hi @BensonChen2000, thanks for starting work on this!

I saw that this PR focuses on ggml_is_contiguous_rows and replaces the contiguous 1D path in the kernel. I've put together an implementation that adds full strided support (handling arbitrary byte-strides nb0..3 with zero CPU fallback across all unary ops) while keeping the fast 1D contiguous kernel intact so standard contiguous inference doesn't take a hit. It's currently passing 172/172 tests on test-backend-ops across FP16 and FP32.

Would you like to collaborate on incorporating these changes into your branch, or would you prefer I open a separate PR?

[BensonChen2000]

Hi @saady789, It would be nice if we collaborate on incorporating these changes into my branch.

[saady789]

@BensonChen2000 have u tried running the test harness and confirmed that non contiguous tensor now run on GPU ??

[JohannesGaessler]

Closing this PR in favor of #29781 .