Skip to content

JIT: recognition of ADC/SBB, MULX/ADCX/ADOX patterns for BigInteger, Decimal, UInt128 - #134039

Open
benaadams wants to merge 5 commits into
dotnet:mainfrom
benaadams:adc
Open

JIT: recognition of ADC/SBB, MULX/ADCX/ADOX patterns for BigInteger, Decimal, UInt128#134039
benaadams wants to merge 5 commits into
dotnet:mainfrom
benaadams:adc

Conversation

@benaadams

@benaadams benaadams commented Sep 16, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Summary

Recognize wide-integer arithmetic expressed in ordinary C# and use hardware carry, borrow, full-width multiplication and division results. No new public API is required.

  • Lower unsigned carry/borrow idioms to ADD/ADC and SUB/SBB on x64, and ADDS/ADCS and SUBS/SBCS on ARM64. Keep carry flags live across eligible counted loops.
  • Recognize multiply-accumulate chains and select MULX/ADCX/ADOX on supported x64 targets. ARM64 uses MUL/UMULH with carry arithmetic. Mark only products belonging to a successfully matched chain.
  • Combine complementary constant shifts into SHRD/EXTR and reuse the hardware remainder for eligible x - (x / y) * y expressions on xarch.
  • Expose these patterns in BigIntegerCalculator, Int128/UInt128, Decimal, internal Number.BigInteger and IEEE decimal helpers. Preserve MulAdd1's four-limb unroll; express Montgomery reduction as a scalar recurrence without a helper call. Match span bounds to loop limits so bounds checks stay outside the main arithmetic loops.
  • Refine register constraints and operand/address reuse for carry arithmetic. Preserve managed byrefs and GC tracking; library loops continue to use spans, without pointer-based rewrites.

The matchers conservatively reject unsupported aliasing, intervening flag clobbers and unsafe movement across calls or exceptions. Carry-local deadness is proved against current control flow, including exception successors, with bounded analysis. Existing fallbacks and architecture-specific 32-bit paths remain available. Register-pressure and consumer-shape guards limit extra moves, spills and flag materialization.

Assembly examples

The excerpts below focus on the arithmetic; unrelated setup, prologues and epilogues are omitted. Register allocation and calling conventions vary by target.

Carry and borrow

For count += (a + b < a) ? 1UL : 0UL, the x64 arithmetic changes from:

; Before
add      rdx, rcx
cmp      rdx, rcx
setb     al
movzx    rax, al
add      r8, rax

; After
add      rcx, rdx
adc      r8, 0

The complete Count64 test method shrinks from 22 to 14 bytes. UInt128 subtraction changes from 40 to 25 bytes on Windows x64:

; Before: low subtraction, borrow materialization, high subtraction
mov      r10, rax
sub      r10, qword ptr [r8]
cmp      r10, rax
seta     al
movzx    rax, al
sub      rdx, qword ptr [r8+0x08]
sub      rdx, rax

; After
sub      rax, qword ptr [r8]
sbb      rdx, qword ptr [r8+0x08]

The corresponding ARM64 arithmetic uses hardware carry/borrow directly:

; Count64
adds     x0, x0, x1
adc      x2, x2, xzr

; UInt128 subtraction
subs     x0, x0, x2
sbc      x1, x1, x3

Reusing subtraction flags in String.IndexOf

In String.IndexOf(char, int), the subtraction computing Length - startIndex supplies the borrow flag for argument validation, eliminating a separate comparison:

 mov      r8d, dword ptr [rcx+0x08]  ; Length
-mov      eax, r8d
-sub      eax, ebx                  ; remaining length
-cmp      r8d, ebx                  ; Length < startIndex?
+sub      r8d, ebx                  ; remaining length + borrow
 jb       throw_out_of_range

The remaining length also stays in the search helper's argument register, removing a move before each of the two call paths. The search implementation is unchanged.

BigInteger multiply-accumulate

Actual BigIntegerCalculator.MulAdd1 on Windows x64 shrinks from 589 bytes / 178 instructions to 344 bytes / 98 instructions. The baseline repeatedly materializes carry with CMP/SETB/MOVZX and spills its running high word. The new four-limb main loop is:

loop:
mov      rcx, rbp
mov      edi, ebx
mulx     r14, rbp, qword ptr [r8+8*rdi]
adcx     rcx, qword ptr [r9+8*rdi]
adox     rcx, rbp
mov      qword ptr [r9+8*rdi], rcx
mulx     rbp, rcx, qword ptr [r8+8*rdi+0x08]
adcx     r14, qword ptr [r9+8*rdi+0x08]
adox     rcx, r14
mov      qword ptr [r9+8*rdi+0x08], rcx
mulx     r14, rcx, qword ptr [r8+8*rdi+0x10]
adcx     rbp, qword ptr [r9+8*rdi+0x10]
adox     rcx, rbp
mov      qword ptr [r9+8*rdi+0x10], rcx
mulx     rbp, rcx, qword ptr [r8+8*rdi+0x18]
adcx     r14, qword ptr [r9+8*rdi+0x18]
adox     rcx, r14
mov      qword ptr [r9+8*rdi+0x18], rcx
lea      ebx, [rbx+0x04]
lea      esi, [rsi-0x01]
mov      ecx, esi
jrcxz    done
jmp      loop
done:

CF and OF are seeded before entry, survive the backedge, and are drained on exit. The loop has no carry spills or per-limb bounds checks. LEA preserves both flags; DEC would destroy OF. JRCXZ plus JMP costs two branch instructions per iteration but preserves both chains. Bounds validation and the scalar tail remain outside this excerpt.

On ARM64, the actual rebuilt helper is 368 bytes / 100 instructions. Its first limb uses:

ldr      x13, [x2, x8, LSL #3]
mul      x14, x13, x4
umulh    x13, x13, x4
ldr      x12, [x0, x8, LSL #3]
adcs     x6, x14, x6
adc      x13, x13, xzr
adds     x6, x6, x12
str      x6, [x0, x8, LSL #3]

The next limb consumes the high word and carry; the final carry also flows into the next loop iteration. This code was executed under local Docker/QEMU ARM64 emulation, including 4,000 random, boundary and overlap cases.

Reusing the hardware remainder

For q = x / y; r = x - q * y; return q + r, the Windows x64 test shrinks from 22 to 15 bytes:

; Before, after setting up RDX:RAX and divisor R8
idiv     r8
imul     r8, rax
sub      rcx, r8
add      rax, rcx

; After
idiv     r8
add      rax, rdx

This recognizes remainder reconstruction. General pairing of separately expressed / and % is not part of this change.

Benchmarks

Fresh measurements compare parent 76e6281d52f with af9cdfc2c6e on an AMD Ryzen 9 9950X, Windows x64, pinned to logical CPU 6. Results are medians from nine fresh-process samples per version, alternating before/after order. Tiering and ReadyToRun are disabled; workstation GC is used. There were no concurrent builds or replays during timing.

Both versions use the same Checked runtime/CoreLib/support framework and separately compiled Release Numerics assemblies from their respective source trees, with the respective Checked JITs. These isolate the JIT + Numerics changes; they are not a comparison of two complete framework distributions or a measurement of the CoreLib Decimal/Int128 rewrites. API compatibility validation was disabled for these isolated benchmark assemblies. Setup, warmup, reflection/delegate creation and explicit collections are outside timing. Checked-runtime overhead remains, including in allocating public operations.

A limb is 64 bits. Public add/subtract/multiply/square use deterministic positive inputs of the stated width; division uses a roughly twice-as-wide dividend. Public result allocation is included. Private helpers use reusable buffers and include delegate overhead. All before/after result digests match, and allocation counts match for every main-harness case. Private multiply helpers allocate nothing.

Representative results (ns/op; speedup = before / after):

Operation Limbs Before After Speedup
MulAdd1 64 66.10 21.57 3.06x
MulAdd1 256 262.27 76.62 3.42x
SubMul1 64 80.47 42.58 1.89x
Mul1 64 63.37 30.50 2.08x
Multiply 64 3,825.36 2,075.68 1.84x
Square 64 4,345.58 2,665.94 1.63x
Add 64 172.29 130.41 1.32x
Subtract 4 57.28 58.05 0.99x
Subtract 64 169.11 127.69 1.32x
DivRem 64 6,998.26 4,350.42 1.61x
Reduce 64 4,347.46 1,873.88 2.32x
ModPow65537 64 167,362.16 87,442.07 1.91x
ModPowDense 64 2,814,806.90 1,386,065.52 2.03x

Four-limb subtraction is approximately flat (1.3% higher median); the larger tested cases improve. Small differences should not be read as statistically established changes. These are warm-cache measurements on one AMD CPU, with no Intel or native ARM64 throughput claim.

Montgomery Reduce includes restoring the input buffer and the private-method delegate call. ModPow65537 uses exponent 65,537; ModPowDense uses 2^256 - 189. Their timings include the complete public operation and allocation. All Montgomery and ModPow result digests match between versions.

All measured cases
Operation Limbs Before ns/op After ns/op Speedup
MulAdd1 3 5.43 4.35 1.25x
MulAdd1 4 5.88 3.85 1.52x
MulAdd1 16 17.01 7.70 2.21x
MulAdd1 64 66.10 21.57 3.06x
MulAdd1 256 262.27 76.62 3.42x
SubMul1 3 5.36 4.45 1.21x
SubMul1 4 6.00 5.16 1.16x
SubMul1 16 20.94 11.78 1.78x
SubMul1 64 80.47 42.58 1.89x
SubMul1 256 316.35 170.90 1.85x
Mul1 3 4.70 4.36 1.08x
Mul1 4 5.38 4.43 1.21x
Mul1 16 14.81 9.67 1.53x
Mul1 64 63.37 30.50 2.08x
Mul1 256 259.80 120.34 2.16x
Multiply 4 86.39 76.57 1.13x
Multiply 16 373.28 212.78 1.75x
Multiply 64 3,825.36 2,075.68 1.84x
Multiply 256 35,292.05 16,833.47 2.10x
Square 4 73.63 61.43 1.20x
Square 16 380.90 258.42 1.47x
Square 64 4,345.58 2,665.94 1.63x
Square 256 40,569.85 24,549.54 1.65x
Add 4 56.18 50.37 1.12x
Add 16 77.41 61.52 1.26x
Add 64 172.29 130.41 1.32x
Add 256 553.21 355.58 1.56x
Subtract 4 57.28 58.05 0.99x
Subtract 16 75.16 70.18 1.07x
Subtract 64 169.11 127.69 1.32x
Subtract 256 556.20 331.86 1.68x
DivRem 4 209.15 192.18 1.09x
DivRem 16 648.31 479.93 1.35x
DivRem 64 6,998.26 4,350.42 1.61x
DivRem 256 69,583.60 39,370.23 1.77x
Reduce 1 15.40 15.17 1.02x
Reduce 2 19.25 18.12 1.06x
Reduce 3 26.64 21.70 1.23x
Reduce 4 35.02 26.19 1.34x
Reduce 8 84.11 52.95 1.59x
Reduce 16 273.59 140.09 1.95x
Reduce 32 1,117.02 536.98 2.08x
Reduce 64 4,347.46 1,873.88 2.32x
Reduce 128 17,300.02 7,228.91 2.39x
ModPow65537 4 1,518.87 1,243.77 1.22x
ModPow65537 8 3,682.53 2,606.80 1.41x
ModPow65537 16 12,357.67 7,488.00 1.65x
ModPow65537 32 46,985.12 25,749.65 1.82x
ModPow65537 64 167,362.16 87,442.07 1.91x
ModPowDense 4 24,317.81 19,641.98 1.24x
ModPowDense 8 64,777.54 42,686.29 1.52x
ModPowDense 16 217,339.38 120,973.32 1.80x
ModPowDense 32 800,926.21 411,956.31 1.94x
ModPowDense 64 2,814,806.90 1,386,065.52 2.03x

All samples, including outliers, are retained in the local measurement artifacts. No samples were discarded. The main harness targets approximately 80 ms per case using parent-calibrated iteration counts shared by both versions; the Montgomery harness uses the same approach.

SuperPMI

Full Windows x64 assembly comparison against the PR parent across all nine locally available collections:

Collection Replayable contexts Assembly differences Smaller Same size Larger Net code bytes
aspire.nativeaot 54,094 80 74 6 0 -2,143
aspnet2.run 39,074 20 20 0 0 -279
benchmarks.run 51,616 45 45 0 0 -1,760
benchmarks.run_pgo 86,553 28 28 0 0 -780
benchmarks.run_pgo_optrepeat 58,091 45 45 0 0 -1,760
libraries.crossgen2 295,996 185 180 5 0 -4,055
libraries.pmi 329,868 160 150 4 6 -2,415
realworld.run 29,872 19 19 0 0 -305
smoke_tests.nativeaot 34,662 177 161 16 0 -4,219
Total 979,826 759 722 31 6 -17,716

The cleaned corpus replays with zero compilation failures and zero missing contexts for both JITs. The original input contains 980,044 contexts; 218 cannot replay against the parent because recorded runtime answers are missing. Those baseline-incomplete records were excluded using the standard mcs -strip cleanup workflow, with their identities retained. No candidate-only failure was excluded. Original collections are unchanged. Collections overlap, so counts and byte totals are per replay context, not unique methods or application-size savings.

Changed contexts shrink from 354,067 to 336,351 generated code bytes (17,716 bytes / 5.00%). This percentage applies only to changed contexts. The six increases total 10 bytes, with a largest increase of four bytes: Int128 conversion/register-allocation tradeoffs and four one-byte differences in dependency-injection helpers. They are not arithmetic compilation failures. Code size alone does not establish throughput.

The parent JIT was rebuilt with only the JIT/EE GUID retargeted to replay this corpus. The ISA addition appends ADX without renumbering existing entries; baseline optimization sources are unchanged. SuperPMI replays the recorded IL and therefore complements, rather than replaces, tests and benchmarks of the modified library sources.

Validation

  • Windows x64 and all cross-target Checked JIT builds; Linux ARM64 Checked runtime/JIT builds.
  • Focused carry, borrow, multiply, division and decimal suites with expensive JIT checks, register stress, AVX2 disabled and hardware intrinsics disabled on x64; ARM64 suites under local Docker/QEMU emulation, including register stress and disabled intrinsics.
  • x64 and ARM64 disassembly assertions for intended instruction sequences.
  • Actual MulAdd1 oracle: 4,000 random, boundary and overlap cases per run; actual subtraction and SubMul1 randomized checks and targeted ARM64 GC-stress checks.
  • IEEE Decimal32/64/128 differential checks: 31,500 arithmetic results matching the comparison baseline, also under register stress.
  • Fresh parent/current benchmark result checks and full x64 corpus comparison reported above.

ARM64 execution and assembly are validated through emulation; native ARM64 timing and a full ARM64 SuperPMI sweep have not been performed. Replay checks compilation and JIT assertions, not runtime arithmetic or GC correctness by itself.

@benaadams
Optimize wide integer arithmetic on x64 and ARM64
af9cdfc 
Recognize unsigned carry and borrow idioms expressed through ordinary
arithmetic without adding managed APIs. Lower eligible straight-line and
counted-loop chains to ADD/ADC and SUB/SBB on x64 and ADDS/ADCS and
SUBS/SBCS on ARM64. Preserve flags across loop backedges and materialize
carry or borrow only where needed.

Recognize full-width multiply-accumulate chains. Add ADX feature detection
and select MULX/ADCX/ADOX for proven products on supported x64 targets;
use MUL/UMULH with carry arithmetic on ARM64. Keep unrelated BigMul nodes
on their existing path. Preserve managed byref tracking and fallback paths,
and refine register constraints, operand reuse, addressing and loop control
to avoid redundant carry spills and loads. Seed ADX without a temporary
register and preserve both carry flags through LEA countdowns and JRCXZ.
Use ordinary DEC selection for loops that need to preserve CF alone.

Combine complementary constant shifts into SHRD or EXTR. Reuse the hardware
remainder for eligible xarch quotient/remainder expressions while preserving
exception behavior and observable destination values. Scope fixed-register
propagation to methods with carry arithmetic and apply conservative
profitability guards. Retain existing select lowering and zero-flag INC/DEC
optimizations when carry transforms would add overhead.

Expose these patterns in Int128, UInt128, Decimal, IEEE decimal helpers and
BigIntegerCalculator. Use matching span bounds for limb loops, retain
MulAdd1 unrolling, and express Montgomery reduction as a scalar carry loop.
Preserve architecture-specific alternatives and 32-bit arithmetic paths.
Use scalar long/ulong Tensor DivRem on x64 to retain the hardware remainder.
Document the source idioms, carry-width requirements and internal helper
preconditions. Widening division requires a quotient that fits in one limb;
span slicing validates destination capacity before entering limb loops.

Prove deleted carry-local values dead from current control flow instead of
relying on liveness computed before lowering. Include exception successors
and promoted-parent aliases, exclude unsupported implicit uses, and bound
analysis work. Use per-query visited bitsets without caching across IR or
control-flow mutations. Skip carry-folding searches until a SETCC may be
available while keeping comparison recognition enabled. Share local-use,
loop-entry, dead-store and address-rewrite helpers. Preserve memory ordering,
flags consumers, LIR use ordering, value lifetimes and control-flow analysis
validity, and require distinct loop exits.

Add correctness and disassembly coverage for carry/borrow chains, wide
products, decimal rounding, division and funnel shifts. Cover boundary and
randomized inputs, struct and field aliases, calls, exceptions, volatile
accesses, arbitrary carry inputs, generic helper expansion and exception-
handler observations.

Validation:
- Windows x64 and all cross-target Checked JIT builds; Linux ARM64 Checked
  runtime and JIT builds.
- x64 arithmetic suites with expensive JIT checks, register stress, AVX2
  disabled and hardware intrinsics disabled.
- Docker/QEMU ARM64 arithmetic suites with expensive checks, register stress
  and hardware intrinsics disabled; targeted x64/ARM64 disassembly checks.
- Actual BigInteger MulAdd1 checks cover 4,000 random, boundary and overlap
  cases per run. Its x64 implementation is 344 bytes and 98 instructions,
  including the ADX loop and MULX scalar tail.
- IEEE Decimal32/64/128 differential checks cover 31,500 arithmetic results,
  matching the comparison baseline byte-for-byte, including under stress.
- Full Windows x64 SuperPMI validation: 980,044 contexts across nine
  overlapping collections, 218 known missing contexts and zero other
  failures. Incremental assembly comparisons found no code-size regressions;
  the final comparison improved one Decimal128 method by nine bytes.
- Changed-line formatting and git diff --check passed.

Compilation replay and code size do not establish execution equivalence or
throughput gains. No full assembly comparison against the parent was rerun
for the final tree, and no ARM64 hardware timing or full ARM64 SuperPMI sweep
was performed.
Copilot AI lite review requested due to automatic review settings September 16, 2026 08:29
@github-actions github-actions Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Sep 16, 2026
@azure-pipelines

azure-pipelines Bot commented Sep 16, 2026

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 8 pipeline(s).
8 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

dotnet-policy-service Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Tagging @dotnet/jit-contrib for JIT-EE GUID update

@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Sep 16, 2026
@dotnet-policy-service

dotnet-policy-service Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Copilot stopped reviewing on behalf of benaadams due to an error September 16, 2026 08:50
Copilot had to stop work due to a timeout.
Copilot AI reviewed Sep 16, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Copilot was unable to run its full agentic suite in this review.

Pull request overview

This PR expands JIT/codegen and framework support for wide/carry-based arithmetic (including ADX on x86/x64), adds targeted lowering/LSRA/emitter updates, and introduces multiple JIT regression tests validating code shape and correctness.

Changes:

  • Add new JIT opt tests covering carry/borrow chains, wide arithmetic, funnel-shift patterns, division/remainder behaviors, and decimal/biginteger helper paths
  • Introduce ADX feature detection and instruction set plumbing (minipal → CoreCLR → R2R/tooling)
  • Refine JIT lowering/codegen/LSRA/emitter behavior for carry arithmetic, funnel shifts, remainder reuse, and map-38 legacy encodings

Reviewed changes

Copilot reviewed 62 out of 63 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/tests/JIT/opt/Add/WideArithmetic.csproj Adds new JIT test project configuration w/ disasm checking
src/tests/JIT/opt/Add/WideArithmetic.cs Adds wide arithmetic correctness + codegen-shape tests (128-bit, carry/borrow)
src/tests/JIT/opt/Add/FunnelShift.csproj Adds new JIT test project configuration w/ disasm checking
src/tests/JIT/opt/Add/FunnelShift.cs Adds funnel-shift pattern tests for x64/arm64 codegen
src/tests/JIT/opt/Add/Division.csproj Adds new JIT test project configuration w/ disasm checking
src/tests/JIT/opt/Add/Division.cs Adds division/remainder and decimal/biginteger helper coverage tests
src/tests/JIT/opt/Add/DecimalWide.csproj Adds new JIT test project configuration for decimal wide helpers
src/tests/JIT/opt/Add/DecimalWide.cs Adds reflection-based tests for internal wide decimal helpers
src/tests/JIT/opt/Add/DecimalMultiply.csproj Adds new JIT test project configuration for decimal multiply
src/tests/JIT/opt/Add/DecimalMultiply.cs Adds decimal multiplication oracle + overflow tests
src/tests/JIT/opt/Add/CarryChains.csproj Adds new JIT test project configuration for carry-chain loops
src/tests/JIT/opt/Add/CarryChains.cs Adds multi-limb carry chain tests emphasizing flags/carry lifetimes
src/tests/JIT/opt/Add/Carry.csproj Adds new JIT test project configuration w/ disasm checking
src/tests/JIT/opt/Add/Carry.cs Adds extensive carry-chain + muladd loop tests + new patterns (funnel shift, etc.)
src/tests/JIT/opt/Add/Borrow.csproj Adds new JIT test project configuration w/ disasm checking
src/tests/JIT/opt/Add/Borrow.cs Adds borrow-chain tests + mul/sub sequences and codegen constraints
src/tests/JIT/opt/Add/ArithmeticCodegen.csproj Adds new JIT test project configuration
src/tests/JIT/opt/Add/ArithmeticCodegen.cs Adds codegen-oriented tests for mul/div remainder, carry across blocks, etc.
src/native/minipal/cpufeatures.h Adds ADX CPU feature flag constant
src/native/minipal/cpufeatures.c Detects ADX via CPUID leaf 7 EBX bit 19
src/libraries/System.Runtime.Numerics/src/System/Numerics/BigIntegerCalculator.PowMod.cs Reworks Montgomery reduce inner loop shape for bounds/carry optimizations
src/libraries/System.Private.CoreLib/src/System/UIntPtr.cs Marks UIntPtr.BigMul AggressiveInlining
src/libraries/System.Private.CoreLib/src/System/UInt128.cs Tweaks DivRem/BigMul for carry/borrow codegen shaping
src/libraries/System.Private.CoreLib/src/System/Numerics/BigIntegerCalculator.Shared.cs Improves limb add/sub and DivRem; reshapes MulAdd1 loop; adds bounds proofs
src/libraries/System.Private.CoreLib/src/System/Number.DecimalIeee754.cs Adds UInt128-specialized wide multiply/add/sub and widening-divide fast paths
src/libraries/System.Private.CoreLib/src/System/Number.BigInteger.cs Fixes/clarifies heuristic division correction comments
src/libraries/System.Private.CoreLib/src/System/Int128.cs Adds 64-bit DivRem fast path using UInt128.DivRem and updates % operator
src/libraries/System.Private.CoreLib/src/System/Decimal.DecCalc.cs Adds widening-divide helper and reshapes decimal division/mul/rounding paths
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorPrimitives.DivRem.cs Disables vectorization for x64 long/ulong to preserve scalar residue behavior
src/coreclr/vm/codeman.cpp Enables InstructionSet_ADX when detected & config allows
src/coreclr/tools/Common/JitInterface/ThunkGenerator/InstructionSetDesc.txt Adds ADX instruction set entries and bumps NEXT_AVAILABLE_R2R_BIT
src/coreclr/tools/Common/JitInterface/CorInfoInstructionSet.cs Adds ADX to instruction-set enums and supported set lists
src/coreclr/tools/Common/Internal/Runtime/ReadyToRunInstructionSetHelper.cs Maps JIT instruction set ADX to R2R instruction set
src/coreclr/tools/Common/Internal/Runtime/ReadyToRunInstructionSet.cs Adds R2R enum value for Adx
src/coreclr/tools/Common/Compiler/HardwareIntrinsicHelpers.cs Adds ADX flag mapping in crossgen tooling
src/coreclr/jit/lsraxarch.cpp LSRA changes for ADX nodes, div/rem pair physreg capture, funnel-shift reg liveness
src/coreclr/jit/lsrabuild.cpp Prevents def reassign for div/rem pair; adds fixed refs for propagated single-reg defs
src/coreclr/jit/lsraarm64.cpp Marks carry ops as flags consumers for allocation decisions
src/coreclr/jit/lsra.h Adds carry-arithmetic state tracking + fixed-def tracking
src/coreclr/jit/lsra.cpp Critical-edge handling for JCMP; selection tweaks when carry arithmetic introduces fixed refs
src/coreclr/jit/lowerxarch.cpp Adds TryLowerDivRem and funnel-shift containment call site
src/coreclr/jit/lowerarmarch.cpp Adds funnel-shift containment + carry-op containment of 0 on ARM64
src/coreclr/jit/lower.h Declares carry-chain lowering pipeline, div/rem reuse, and funnel-shift containment
src/coreclr/jit/liveness.cpp Ensures div/rem pair division isn’t removed; null-guards prev flag clearing; adds ADX seed handling
src/coreclr/jit/instrsxarch.h Adds ADCX/ADOX and JRCXZ encodings
src/coreclr/jit/inductionvariableopts.cpp Adds carry-loop-aware countdown heuristics + folds four-limb offsets
src/coreclr/jit/hwintrinsiccodegenxarch.cpp Adds MULX codegen path for matched multiply-carry patterns
src/coreclr/jit/hwintrinsic.cpp Adds ADX slot to ISA range table
src/coreclr/jit/gtlist.h Adds new GT nodes for carry/borrow/ADX chain representation
src/coreclr/jit/gentree.h Adds funnel-shift/divrem-pair helpers and flag-consumption tracking
src/coreclr/jit/gentree.cpp Implements IsFunnelShift and updates use-edge iteration / node display for new GT nodes
src/coreclr/jit/emitxarch.h Adds helper to classify legacy map-38 instructions
src/coreclr/jit/emitxarch.cpp Adjusts encoding/size/output paths for JRCXZ and legacy map-38 instructions
src/coreclr/jit/emitarm64.cpp Allows ZR as the third operand for adc/sbc class instructions
src/coreclr/jit/compiler.h Declares optFoldFourLimbOffsets hook
src/coreclr/jit/codegenxarch.cpp Adds funnel-shift codegen, carry op support, ADX drain/seed, JCMP jrcxz lowering
src/coreclr/jit/codegenarmarch.cpp Enables codegen dispatch for carry/borrow ops on ARM64
src/coreclr/jit/codegenarm64.cpp Implements carry/borrow ops with adc/sbc/cinc and funnel-shift via extr
src/coreclr/inc/readytoruninstructionset.h Adds READYTORUN_INSTRUCTION_Adx
src/coreclr/inc/jiteeversionguid.h Updates JIT/EE version GUID
src/coreclr/inc/corinfoinstructionset.h Adds ADX to CORINFO_InstructionSet + string/R2R mappings
src/coreclr/inc/clrconfigvalues.h Adds EXTERNAL_EnableADX config knob
Suppressed comments (4)

src/coreclr/jit/codegenxarch.cpp:1

  • The funnel-shift codegen path will generate incorrect code if LSRA assigns targetReg == hi while lo is in a different register (the mov targetReg, lo would clobber hi before shrd consumes it). The current assert(targetReg != hi) prevents this in debug, but release builds would still be at risk if the allocation happens. Consider making the codegen robust by handling targetReg == hi (e.g., emit the equivalent shld form with the complementary count, or ensure the destination is forced/preferred to lo and guaranteed not to pick hi), rather than relying on an assertion.
// Licensed to the .NET Foundation under one or more agreements.

src/coreclr/tools/Common/JitInterface/CorInfoInstructionSet.cs:1

  • The InstructionSetInfo entries for adx provide an empty managed name (second argument). If this field is used by tooling/diagnostics (e.g., for display, option names, or managed wrapper association), leaving it empty can cause confusing output or missing metadata. Consider supplying an appropriate name (or adding a comment explaining why ADX intentionally has no managed-facing name).
    src/coreclr/tools/Common/JitInterface/CorInfoInstructionSet.cs:1
  • The InstructionSetInfo entries for adx provide an empty managed name (second argument). If this field is used by tooling/diagnostics (e.g., for display, option names, or managed wrapper association), leaving it empty can cause confusing output or missing metadata. Consider supplying an appropriate name (or adding a comment explaining why ADX intentionally has no managed-facing name).
    src/coreclr/jit/gentree.cpp:1
  • GenTree::IsFunnelShift() currently classifies any contained OR(RSZ, LSH) as a funnel shift without validating that the shift counts are immediate constants (and within the legal range for extr on ARM64) or that they are the complementary counts expected by the codegen paths. Since xarch/arm64 codegen assumes an immediate (AsIntCon()->IconValue()), tightening this predicate (e.g., requiring constant shift counts and/or a canonicalized form) would make the invariant explicit and reduce the risk of assertion failures or invalid immediates.

Comment thread src/coreclr/jit/emitxarch.cpp Outdated
Comment thread src/coreclr/jit/lowerxarch.cpp
Copilot AI review requested due to automatic review settings September 16, 2026 08:57
Copilot AI reviewed Sep 16, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The broad JIT, runtime, library, and architecture-specific changes require final human review.

Review details
  • Files reviewed: 62/63 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings September 16, 2026 09:30
@benaadams benaadams changed the title Optimize wide integer arithmetic on x64 and ARM64 JIT: recognition of ADC/SBB, MULX/ADCX/ADOX patterns for BigInteger, Decimal, UInt128 Sep 16, 2026
Copilot stopped reviewing on behalf of benaadams due to an error September 16, 2026 09:50
Copilot had to stop work due to a timeout.
Copilot AI reviewed Sep 16, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Copilot was unable to run its full agentic suite in this review.

Pull request overview

Copilot reviewed 62 out of 63 changed files in this pull request and generated no new comments.

Suppressed comments (4)

src/tests/JIT/opt/Add/WideArithmetic.cs:1

  • This new test file is missing the standard .NET Foundation MIT license header present in other newly added tests (e.g., FunnelShift.cs, Division.cs). Add the usual header block at the top of the file to match repository conventions.
    src/tests/JIT/opt/Add/Borrow.cs:1
  • This new test file is missing the standard .NET Foundation MIT license header used across the repo. Add the header comment block at the top of the file for consistency and compliance.
    src/tests/JIT/opt/Add/DecimalMultiply.cs:1
  • This new test file is missing the standard .NET Foundation MIT license header. Other new tests in this PR include it; please add the same header here.
    src/libraries/System.Private.CoreLib/src/System/Decimal.DecCalc.cs:1
  • The remainder computation relies on modulo-2^32 wraparound (low - quotient * den) rather than directly expressing dividend - quotient * den. While this works given the method’s contract (remainder fits in 32 bits), it’s non-obvious and easy to misread as a bug. Consider computing the remainder from dividend explicitly (e.g., using 64-bit arithmetic and casting at the end) or add an explanatory comment tying correctness to the invariants (high < den, remainder < den < 2^32).

Copilot AI review requested due to automatic review settings September 16, 2026 10:11
Copilot AI reviewed Sep 16, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The broad JIT, runtime, and numeric-library changes require final human review.

Review details
  • Files reviewed: 62/63 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@MichalPetryka

MichalPetryka commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

The API proposal variants that return both the value and carry probably shouldnt be closed since there are usecases that need both, like with divrem.

@benaadams

benaadams commented Sep 16, 2026

Copy link
Copy Markdown
Member Author

The API proposal variants that return both the value and carry probably shouldnt be closed since there are usecases that need both, like with divrem.

#76502 and #82194 are only marked as "Partially addresses"/"addressed" which means its not recognised as closing by github

This was referenced Sep 16, 2026
Open
Closed
Closed
This was referenced Sep 16, 2026
Open
Closed
Open
Open
Copilot AI review requested due to automatic review settings September 16, 2026 22:24
Copilot AI reviewed Sep 16, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The changes span core JIT lowering/LSRA/emitter plus ISA plumbing and numerics codegen-shaping, which warrants final human review despite only minor actionable nits found.

Review details
  • Files reviewed: 62/63 changed files
  • Comments generated: 2
  • Review effort level: Lite

Comment thread src/coreclr/jit/inductionvariableopts.cpp
Comment on lines +1264 to +1267
// SCEV's general trip-count materialization currently handles only unit
// strides. Recognize the bounded four-lane case: i starts at zero, steps
// by four, and continues while i < length - 3 for a nonnegative length.
// Its trip count is length / 4; the induction variable cannot overflow.
Comment thread src/libraries/System.Private.CoreLib/src/System/Decimal.DecCalc.cs
Comment on lines +518 to +525
prod1 = bufDen.Low64;
do
{
quo--;
num += prod1;
high += den;
high += (num < prod1) ? 1L : 0L;
} while (high < 0);
This was referenced Sep 17, 2026
Open
Open
Open
@tannergooding

tannergooding commented Sep 21, 2026

Copy link
Copy Markdown
Member

This is another massive PR that likely needs to be split up and done incrementally (there's at least 3, if not 4+, different PRs here)

I'd note I have my own local branch that is handling some adc/sbb lightup as well, which went a bit of a different route

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@MichalStrehovsky MichalStrehovsky Awaiting requested review from MichalStrehovsky MichalStrehovsky is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Milestone

No milestone

4 participants

@benaadams @MichalPetryka @tannergooding