Fix discarding of NRE's when unboxing nullables in Mono and JIT

[jakobbotsch]

VN-based constant folding did not preserve exceptions from the root node. For GetType folding, that results in discarding an explicit nullcheck.

Retain throwing roots when their VN exception sets are not covered by their operands. Fold unused GetType operations to null checks through shared side-effect extraction, and add a targeted nullable GetType regression test.

Mono had a similar issue for nullables only, which was fixed in #132732 but later reverted. This PR includes that fix.

Fix #133207

[azure-pipelines]

Azure Pipelines:
Successfully started running 5 pipeline(s).
11 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[jakobbotsch]

/azp run runtime-coreclr superpmi-diffs

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[jakobbotsch]

/ba-g Failure is #130439

[jakobbotsch]

cc @dotnet/jit-contrib PTAL @EgorBo

[azure-pipelines]

Azure Pipelines:
Successfully started running 5 pipeline(s).
11 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟡 Changes recommended

The new exception-set subset logic unions operand VNs without guarding against undefined VN pairs, which can trigger debug asserts/crashes in some loop/PHI scenarios.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR fixes a JIT and Mono interpreter optimization bug where VN-based folding (notably GetType folding on boxed nullables) could drop required faulting behavior, leading to missing NullReferenceExceptions. It also adds a targeted JIT regression test to prevent recurrence.

Changes:

• Update JIT VN-based constant folding to retain “throwing roots” when their exception sets aren’t covered by operands, and improve side-effect extraction so unused GetType can be reduced to a null-check.
• Reapply the Mono interpreter fix to avoid optimizing box Nullable<T>; call GetType() into a constant type lookup.
• Add a regression test for Nullable<T>.GetType() behavior (throws on null; returns underlying type when non-null).

File summaries

File	Description
src/coreclr/jit/assertionprop.cpp	Adjust VN-based folding to decide when to preserve root side effects based on exception-set coverage.
src/coreclr/jit/gentree.cpp	Add helper to turn unused GetType into a NULLCHECK via shared side-effect extraction.
src/coreclr/jit/compiler.h	Declare the new side-effect extraction helper on Compiler.
src/mono/mono/mini/interp/transform.c	Disable the box; GetType optimization for nullable types in the interpreter.
src/tests/JIT/Regression/JitBlue/Runtime_133207/Runtime_133207.cs	Add xUnit regression coverage for nullable GetType throwing/returning behavior.
src/tests/JIT/Regression/Regression_ro_2.csproj	Include the new regression test in the JIT regression project.

Review details

• Files reviewed: 6/6 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

It changes JIT optimization and side-effect extraction behavior in broadly-used code paths, so it warrants final maintainer review despite the added regression coverage.

Review details

• Files reviewed: 6/6 changed files
• Comments generated: 0 new
• Review effort level: Lite

[jakobbotsch]

Minor diffs.

[jakobbotsch]

/backport to release/11.0

[github-actions]

Started backporting to release/11.0 (link to workflow run)

[github-actions]

@jakobbotsch backporting to release/11.0 failed, the patch most likely resulted in conflicts. Please backport manually!

git am output

$ git cherry-pick bc3f1a51d9fad655cb3fbf28b94e98fceba7b890

Auto-merging src/coreclr/jit/assertionprop.cpp
Auto-merging src/coreclr/jit/compiler.h
Auto-merging src/coreclr/jit/gentree.cpp
Auto-merging src/tests/JIT/Regression/Regression_ro_2.csproj
CONFLICT (content): Merge conflict in src/tests/JIT/Regression/Regression_ro_2.csproj
error: could not apply bc3f1a51d9f... Fix discarding of NRE's when unboxing nullables in Mono and JIT (#133351)
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
hint: Disable this message with "git config set advice.mergeConflict false"


$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch

Applying: JIT: preserve root exceptions during VN constant folding
Using index info to reconstruct a base tree...
M	src/coreclr/jit/assertionprop.cpp
M	src/coreclr/jit/compiler.h
M	src/coreclr/jit/gentree.cpp
M	src/tests/JIT/Regression/Regression_ro_2.csproj
Falling back to patching base and 3-way merge...
Auto-merging src/coreclr/jit/assertionprop.cpp
Auto-merging src/coreclr/jit/compiler.h
Auto-merging src/coreclr/jit/gentree.cpp
Auto-merging src/tests/JIT/Regression/Regression_ro_2.csproj
CONFLICT (content): Merge conflict in src/tests/JIT/Regression/Regression_ro_2.csproj
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 JIT: preserve root exceptions during VN constant folding
Error: The process '/usr/bin/git' failed with exit code 128

Link to workflow output