Repository navigation
[flang][CUDA] Emit ignore_denormal_mode for CUDA Fortran atomicadd - #232726
Merged
chsigg merged 2 commits intoOct 18, 2026
Merged
Conversation
llvmorg-github-actions
Bot
added
flang
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
|
@llvm/pr-subscribers-flang-fir-hlfir Author: Christian Sigg (chsigg) ChangesFull diff: https://github.com/llvm/llvm-project/pull/232726.diff 3 Files Affected:
diff --git a/flang/lib/Optimizer/Builder/CUDAIntrinsicCall.cpp b/flang/lib/Optimizer/Builder/CUDAIntrinsicCall.cpp
index e979bbc44af98f..4e61335079711e 100644
--- a/flang/lib/Optimizer/Builder/CUDAIntrinsicCall.cpp
+++ b/flang/lib/Optimizer/Builder/CUDAIntrinsicCall.cpp
@@ -687,8 +687,11 @@ static mlir::Value genAtomBinOp(fir::FirOpBuilder &builder, mlir::Location &loc,
mlir::Value arg1) {
auto llvmPointerType = mlir::LLVM::LLVMPointerType::get(builder.getContext());
arg0 = builder.createConvert(loc, llvmPointerType, arg0);
- return mlir::LLVM::AtomicRMWOp::create(builder, loc, binOp, arg0, arg1,
- mlir::LLVM::AtomicOrdering::seq_cst);
+ bool ignoreDenormalMode = binOp == mlir::LLVM::AtomicBinOp::fadd;
+ return mlir::LLVM::AtomicRMWOp::create(
+ builder, loc, binOp, arg0, arg1, mlir::LLVM::AtomicOrdering::seq_cst,
+ /*syncscope=*/llvm::StringRef(), /*alignment=*/0, /*isVolatile=*/false,
+ /*ignoreDenormalMode=*/ignoreDenormalMode);
}
// ATOMICADD
diff --git a/flang/test/Lower/CUDA/cuda-atomicadd.cuf b/flang/test/Lower/CUDA/cuda-atomicadd.cuf
index dff029d33930d4..39e5a56808a707 100644
--- a/flang/test/Lower/CUDA/cuda-atomicadd.cuf
+++ b/flang/test/Lower/CUDA/cuda-atomicadd.cuf
@@ -8,7 +8,7 @@ attributes(global) subroutine test_atomicaddvector_r2()
end subroutine
! CHECK-LABEL: func.func @_QPtest_atomicaddvector_r2() attributes {cuf.proc_attr = #cuf.cuda_proc<global>{{.*}}}
-! CHECK: llvm.atomicrmw fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, vector<2xf16>
+! CHECK: llvm.atomicrmw ignore_denormal_mode fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, vector<2xf16>
attributes(global) subroutine test_atomicaddvector_r4()
real(4), device :: a(2), tmp1(2), tmp2(2)
@@ -16,7 +16,7 @@ attributes(global) subroutine test_atomicaddvector_r4()
end subroutine
! CHECK-LABEL: func.func @_QPtest_atomicaddvector_r4() attributes {cuf.proc_attr = #cuf.cuda_proc<global>{{.*}}}
-! CHECK: llvm.atomicrmw fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, vector<2xf32>
+! CHECK: llvm.atomicrmw ignore_denormal_mode fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, vector<2xf32>
attributes(global) subroutine test_atomicadd_r2x4()
real(4), device :: a(2), tmp1(2), tmp2(2)
@@ -24,7 +24,7 @@ attributes(global) subroutine test_atomicadd_r2x4()
end subroutine
! CHECK-LABEL: func.func @_QPtest_atomicadd_r2x4() attributes {cuf.proc_attr = #cuf.cuda_proc<global>{{.*}}}
-! CHECK: llvm.atomicrmw fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, vector<2xf32>
+! CHECK: llvm.atomicrmw ignore_denormal_mode fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, vector<2xf32>
attributes(global) subroutine test_atomicadd_r4x4()
real(4), device :: a(4), tmp1(4), tmp2(4)
@@ -33,3 +33,27 @@ end subroutine
! CHECK-LABEL: func.func @_QPtest_atomicadd_r4x4() attributes {cuf.proc_attr = #cuf.cuda_proc<global>{{.*}}}
! CHECK: atom.add.v4.f32
+
+attributes(global) subroutine test_atomicadd_r4()
+ real(4), device :: a, tmp1, tmp2
+ tmp1 = atomicadd(a, tmp2)
+end subroutine
+
+! CHECK-LABEL: func.func @_QPtest_atomicadd_r4() attributes {cuf.proc_attr = #cuf.cuda_proc<global>{{.*}}}
+! CHECK: llvm.atomicrmw ignore_denormal_mode fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, f32
+
+attributes(global) subroutine test_atomicadd_r8()
+ real(8), device :: a, tmp1, tmp2
+ tmp1 = atomicadd(a, tmp2)
+end subroutine
+
+! CHECK-LABEL: func.func @_QPtest_atomicadd_r8() attributes {cuf.proc_attr = #cuf.cuda_proc<global>{{.*}}}
+! CHECK: llvm.atomicrmw ignore_denormal_mode fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, f64
+
+attributes(global) subroutine test_atomicadd_i4()
+ integer(4), device :: a, tmp1, tmp2
+ tmp1 = atomicadd(a, tmp2)
+end subroutine
+
+! CHECK-LABEL: func.func @_QPtest_atomicadd_i4() attributes {cuf.proc_attr = #cuf.cuda_proc<global>{{.*}}}
+! CHECK: llvm.atomicrmw add %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, i32
diff --git a/flang/test/Lower/CUDA/cuda-device-proc.cuf b/flang/test/Lower/CUDA/cuda-device-proc.cuf
index a1c3c33a76811d..b9081681569be9 100644
--- a/flang/test/Lower/CUDA/cuda-device-proc.cuf
+++ b/flang/test/Lower/CUDA/cuda-device-proc.cuf
@@ -123,9 +123,9 @@ end
! CHECK: %{{.*}} = nvvm.barrier.reduction #nvvm.reduction<or> %[[CONV]] -> i32
! CHECK: %{{.*}} = llvm.atomicrmw add %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, i32
! CHECK: %{{.*}} = llvm.atomicrmw add %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, i64
-! CHECK: %{{.*}} = llvm.atomicrmw fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, f32
-! CHECK: %{{.*}} = llvm.atomicrmw fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, f64
-! CHECK: %{{.*}} = llvm.atomicrmw fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, vector<2xf16>
+! CHECK: %{{.*}} = llvm.atomicrmw ignore_denormal_mode fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, f32
+! CHECK: %{{.*}} = llvm.atomicrmw ignore_denormal_mode fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, f64
+! CHECK: %{{.*}} = llvm.atomicrmw ignore_denormal_mode fadd %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, vector<2xf16>
! CHECK: %{{.*}} = llvm.atomicrmw sub %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, i32
! CHECK: %{{.*}} = llvm.atomicrmw sub %{{.*}}, %{{.*}} seq_cst : !llvm.ptr, i64
|
Contributor
Author
Merge activity
|
chsigg
force-pushed
the
users/chsigg/nvvm-autoupgrade-atomic-ignore-denormal-mode
branch
6 times, most recently
from
October 17, 2026 07:29
1366bc8 to
6b8a837
Compare
chsigg
force-pushed
the
users/chsigg/nvvm-autoupgrade-atomic-ignore-denormal-mode
branch
from
October 17, 2026 15:55
6b8a837 to
0c314a6
Compare
chsigg
force-pushed
the
users/chsigg/flang-cuda-atomic-ignore-denormal-mode
branch
from
October 17, 2026 15:55
f382e48 to
69eddc6
Compare
Base automatically changed from
users/chsigg/nvvm-autoupgrade-atomic-ignore-denormal-mode
to
main
October 17, 2026 16:44
chsigg
deleted the
users/chsigg/flang-cuda-atomic-ignore-denormal-mode
branch
October 18, 2026 06:37
chsigg
added a commit
that referenced
this pull request
Oct 18, 2026
This restores the strict FTZ default originally introduced in #200732 and temporarily relaxed in #206154. Frontends can now explicitly opt in to native FTZ-mismatched atomics on a per-instruction basis using `!atomic.ignore.denormal.mode` metadata (#217585, #217586, #217587, #217588, #232725, #232726), which is supported in Clang, Flang, MLIR, and XLA. Users who need the previous behavior globally can still pass `-nvptx-allow-ftz-atomics=true`. Update test cases to verify the new default disallow behavior.
alanzhao1
pushed a commit
to alanzhao1/llvm-project
that referenced
this pull request
Oct 25, 2026
…31429) This restores the strict FTZ default originally introduced in llvm#200732 and temporarily relaxed in llvm#206154. Frontends can now explicitly opt in to native FTZ-mismatched atomics on a per-instruction basis using `!atomic.ignore.denormal.mode` metadata (llvm#217585, llvm#217586, llvm#217587, llvm#217588, llvm#232725, llvm#232726), which is supported in Clang, Flang, MLIR, and XLA. Users who need the previous behavior globally can still pass `-nvptx-allow-ftz-atomics=true`. Update test cases to verify the new default disallow behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CUDA Fortran
atomicaddhas the same device semantics as CUDA C++atomicAdd(), targeting PTXatom.add(which flushes f32 denormals to zero on global memory).Set
ignore_denormal_modeonmlir::LLVM::AtomicRMWOpwhen lowering CUDA Fortranatomicaddfor floating-point types ingenAtomBinOp, matching Clang's CUDA/HIPatomicAddlowering (#217587) so it continues to emit nativeatom.add.f32when-nvptx-allow-ftz-atomicsdefaults tofalse(#231429).