[maleadt]

This adds a fastmath option to CUDABackend, so that KernelAbstractions kernels can opt into the same kernel-wide floating-point relaxations as @cuda fastmath=true: fast-math flags on every floating-point instruction, less precise square roots, and flushing of Float32 subnormals to sign-preserving zero.

using CUDA, KernelAbstractions

@kernel function axpy!(y, a, x)
    i = @index(Global)
    @inbounds y[i] += a * x[i]
end

kernel! = axpy!(CUDABackend(fastmath=true), 256)
kernel!(y, 2f0, x; ndrange=length(y))

Like @cuda, the default follows Julia's --math-mode setting. The existing positional CUDABackend(prefer_blocks, always_inline) constructor keeps working, and the option is also forwarded to the compiler job used by Enzyme (EnzymeCore.compiler_job_from_backend).

An earlier version of this PR motivated the option with global-memory Float32 atomic additions, which LLVM 23.1.1 lowered to a CAS loop in IEEE mode. That no longer applies: NVPTX_LLVM_Backend_jll 23.1.2 backports llvm/llvm-project#206154, so these atomics lower to the native (subnormal-flushing) atom.add.f32 regardless of fast math, and the atomics tests and documentation have been dropped accordingly.

[maleadt]

Oh wait, I just saw llvm/llvm-project#206154 now. I'll just backport that to fix the performance regression, but this PR seems useful regardless to give people full control.

[github-actions]

CUDA.jl Benchmarks

Details

Benchmark suite	Current: 8b6f3bb	Previous: 9667f8b	Ratio
array/accumulate/Float32/1d	99115 ns	99026 ns	1.00
array/accumulate/Float32/dims=1	71590 ns	71879 ns	1.00
array/accumulate/Float32/dims=1L	1589137 ns	1590250 ns	1.00
array/accumulate/Float32/dims=2	137631 ns	138174 ns	1.00
array/accumulate/Float32/dims=2L	662425 ns	662680 ns	1.00
array/accumulate/Int64/1d	118506 ns	118998 ns	1.00
array/accumulate/Int64/dims=1	75209 ns	75609 ns	0.99
array/accumulate/Int64/dims=1L	1700173 ns	1699629 ns	1.00
array/accumulate/Int64/dims=2	150054 ns	150163 ns	1.00
array/accumulate/Int64/dims=2L	987805 ns	988632 ns	1.00
array/broadcast	17993 ns	18282 ns	0.98
array/broadcast launch	9404 ns	9520 ns	0.99
array/construct	880.2 ns	864.8620689655172 ns	1.02
array/copy	16774 ns	16708 ns	1.00
array/copyto!/cpu_to_gpu	208818 ns	209517 ns	1.00
array/copyto!/gpu_to_cpu	240370 ns	241910 ns	0.99
array/copyto!/gpu_to_gpu	8947 ns	9008.666666666666 ns	0.99
array/iteration/findall/bool	133482 ns	134068 ns	1.00
array/iteration/findall/int	146456 ns	146861 ns	1.00
array/iteration/findfirst/bool	69005 ns	69837 ns	0.99
array/iteration/findfirst/int	70896 ns	71519 ns	0.99
array/iteration/findmin/1d	66668 ns	67457 ns	0.99
array/iteration/findmin/2d	99851 ns	100165 ns	1.00
array/iteration/logical	192741 ns	192387 ns	1.00
array/iteration/scalar	63739 ns	62433 ns	1.02
array/permutedims/2d	48653 ns	48726 ns	1.00
array/permutedims/3d	49842 ns	50366 ns	0.99
array/permutedims/4d	49921 ns	49958 ns	1.00
array/random/rand/Float32	11498 ns	11818 ns	0.97
array/random/rand/Int64	21503 ns	21660 ns	0.99
array/random/rand!/Float32	7731.25 ns	7798.75 ns	0.99
array/random/rand!/Int64	18827 ns	18839 ns	1.00
array/random/randn/Float32	33118 ns	33747 ns	0.98
array/random/randn!/Float32	25163 ns	23454 ns	1.07
array/reductions/mapreduce/Float32/1d	32184 ns	32557 ns	0.99
array/reductions/mapreduce/Float32/dims=1	37510 ns	37955 ns	0.99
array/reductions/mapreduce/Float32/dims=1L	50891 ns	50851 ns	1.00
array/reductions/mapreduce/Float32/dims=2	56308 ns	56324 ns	1.00
array/reductions/mapreduce/Float32/dims=2L	66697 ns	67048 ns	0.99
array/reductions/mapreduce/Int64/1d	39436 ns	40294 ns	0.98
array/reductions/mapreduce/Int64/dims=1	40314 ns	40454 ns	1.00
array/reductions/mapreduce/Int64/dims=1L	88639 ns	88452 ns	1.00
array/reductions/mapreduce/Int64/dims=2	57926 ns	58049 ns	1.00
array/reductions/mapreduce/Int64/dims=2L	83267 ns	83379 ns	1.00
array/reductions/reduce/Float32/1d	32328 ns	32640 ns	0.99
array/reductions/reduce/Float32/dims=1	37472 ns	37584 ns	1.00
array/reductions/reduce/Float32/dims=1L	50525 ns	50757 ns	1.00
array/reductions/reduce/Float32/dims=2	55103 ns	55245 ns	1.00
array/reductions/reduce/Float32/dims=2L	67178 ns	67643 ns	0.99
array/reductions/reduce/Int64/1d	39152 ns	40323 ns	0.97
array/reductions/reduce/Int64/dims=1	40401 ns	40550 ns	1.00
array/reductions/reduce/Int64/dims=1L	88321 ns	88506 ns	1.00
array/reductions/reduce/Int64/dims=2	57863 ns	58182 ns	0.99
array/reductions/reduce/Int64/dims=2L	82715 ns	83616 ns	0.99
array/reverse/1d	16793 ns	16830 ns	1.00
array/reverse/1dL	69581 ns	69663 ns	1.00
array/reverse/1dL_inplace	67350 ns	67343 ns	1.00
array/reverse/1d_inplace	8503 ns	8432 ns	1.01
array/reverse/2d	19839 ns	20256 ns	0.98
array/reverse/2dL	73153 ns	73457 ns	1.00
array/reverse/2dL_inplace	67169 ns	67287 ns	1.00
array/reverse/2d_inplace	11955 ns	12071 ns	0.99
array/sorting/1d	2643649 ns	2636312 ns	1.00
array/sorting/2d	1017821 ns	1018165 ns	1.00
array/sorting/by	3172797 ns	3173659 ns	1.00
cuda/synchronization/context/auto	1029.5 ns	1001.6 ns	1.03
cuda/synchronization/context/blocking	770.8181818181819 ns	775.8265306122449 ns	0.99
cuda/synchronization/context/nonblocking	5666.166666666667 ns	5855.833333333333 ns	0.97
cuda/synchronization/stream/auto	864.6981132075472 ns	857.3333333333334 ns	1.01
cuda/synchronization/stream/blocking	662.6754966887418 ns	663.547770700637 ns	1.00
cuda/synchronization/stream/nonblocking	5625.285714285715 ns	5734 ns	0.98
integration/byval/reference	147883 ns	147855 ns	1.00
integration/byval/slices=1	149155 ns	149029 ns	1.00
integration/byval/slices=2	291965 ns	292135 ns	1.00
integration/byval/slices=3	435273 ns	435065 ns	1.00
integration/cudadevrt	104942 ns	104967 ns	1.00
integration/volumerhs	9146585 ns	9148328 ns	1.00
kernel/indexing	12850 ns	13029 ns	0.99
kernel/indexing_checked	13410 ns	13711 ns	0.98
kernel/launch	2024.3333333333333 ns	2015.3333333333333 ns	1.00
kernel/occupancy	682.6644295302013 ns	667.0967741935484 ns	1.02
kernel/rand	13770 ns	17377 ns	0.79
latency/import	4208326298 ns	4219408633 ns	1.00
latency/precompile	4982924558 ns	4993629185 ns	1.00
latency/ttfp	4677043195 ns	4666797490 ns	1.00

This comment was automatically generated by workflow using github-action-benchmark.

[maleadt]

CI failure unrelated, will be handled by #3283.