JIT: (bug) range analysis treats int<->float casts as value-preserving

[EgorBo]

Range analysis reuses the cast source range for casts to/from floating-point types, ignoring rounding and saturation. This leads to wrong relop folding and a removed bounds check (out-of-bounds read).

Minimal Repro

using System;
using System.Runtime.CompilerServices;

public class Program
{
    public static void Main()
    {
        Console.WriteLine(Test(16777219));
        Console.WriteLine(Test2(-5));
        try { Console.WriteLine(Test3(new byte[16777220], 16777219)); }
        catch (IndexOutOfRangeException) { Console.WriteLine("IOORE"); }
    }

    [MethodImpl(MethodImplOptions.NoInlining | MethodImplOptions.AggressiveOptimization)]
    static int Test(int x)
    {
        if (x >= 0 && x <= 16777219)
        {
            int y = (int)(float)x; // 16777220
            if (y > 16777219) return 1;
        }
        return 0;
    }

    [MethodImpl(MethodImplOptions.NoInlining | MethodImplOptions.AggressiveOptimization)]
    static int Test2(int x)
    {
        if (x < 0)
        {
            uint u = (uint)(double)x; // saturates to 0
            if (u == 0) return 1;
        }
        return 0;
    }

    [MethodImpl(MethodImplOptions.NoInlining | MethodImplOptions.AggressiveOptimization)]
    static int Test3(byte[] arr, int x)
    {
        if (arr.Length == 16777220 && x >= 0 && x <= 16777219)
            return arr[(int)(float)x]; // index 16777220 -> must throw
        return -1;
    }
}

Expected

1
1
IOORE

Actual

0
0
0

Test3 has no bounds check and reads arr[16777220].

Regression?

Yes, regressed in .NET 11: .NET 9 and 10 are correct; .NET 11 RC2 and main are affected (win-x64).

Notes

In GetRangeFromAssertionsWorker (VNF_Cast case), result = castOpRange is only valid for integral-to-integral casts; for float/double source or target the source range doesn't bound the result.
Also TYP_UINT is mapped to the TYP_INT range, so [INT_MIN..-1] is propagated to a value that is actually 0.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.