[wasm] Generate thunks for unmanaged calli signatures and allow thiscall by lewing · Pull Request #134826 · dotnet/runtime

lewing · 2026-09-29T00:43:43Z

On browser CoreCLR, JIT/Directed/callconv/{Cdecl,StdCall,PlatformDefault}MemberFunction and ThisCall fail with a GetCookieForCalliSig: unknown thunk signature assert. The MemberFunction tests print WASM calli missing for key: MS8ii, which comes from delegate* unmanaged[Cdecl, MemberFunction]<C*, int, SizeF>. That signature only shows up as a calli, so the thunk generator never sees it. ThisCall gets rejected by the callconv switch before a key is even built.

#133571 disabled these tests on browser. This PR fixes the cause and re-enables all four tests.

Changes

PortableCallHelpers/PInvokeCollector.cs: the new CollectUnmanagedCalliSignatures scans IL for calli through unmanaged function pointers. It adds their signatures to the interp-to-native thunk table. Managed and varargs calli, generic-shaped signatures, and signatures that can't be lowered are skipped, with a Verbose log.
JitInterface/WasmLowering.cs: GetSignature treated UnmanagedCallingConvention (0x9) as a bit flag, so Cdecl, StdCall and ThisCall signatures (0x1–0x3) were lowered as managed. It now checks the masked calling convention, and the collector no longer has to pass IsUnmanagedCallersOnly to compensate.
vm/wasm/helpers.cpp: ComputeCalliSigThunk now accepts IMAGE_CEE_CS_CALLCONV_THISCALL.
PortableCallHelpers/PInvokeTableGenerator.cs: when a reverse thunk returns a struct that the wasm C ABI returns by reference, the thunk now takes the hidden leading sret pointer and hands it to the interpreter as the return buffer. This applies to [UnmanagedCallersOnly] wrappers and exports. Before, it was declared as returning void *. Native code calling GetSize through the vtable (SizeF return) then trapped with function signature mismatch, and the interpreter would have written 8 bytes into a 4-byte local. The P/Invoke declaration path already handled this. This also removes the rejection of callbacks that return through a hidden buffer, which [wasm][coreclr] Dispatch R2R-compiled UnmanagedCallersOnly callbacks to their native entrypoint #134355 added because the wrapper didn't model that ABI yet (cc @pavelsavara). The export-only R2R dispatch from [wasm][coreclr] Dispatch R2R-compiled UnmanagedCallersOnly callbacks to their native entrypoint #134355 forwards sret the same way.
WasmArgumentLayoutTests.cs: new tests PortableCallHelpersGeneratorEmitsThunksForUnmanagedCalliSites, PortableCallHelpersGeneratorReturnsStructsThroughHiddenPointerInReverseThunks and SignatureCallingConventionSelectsLowering.
src/tests/JIT/Directed/callconv/: reverts the ActiveIssue, WasmBuildTestCorerun=false, and CLRTestTargetUnsupported disables that Fix combined ActiveIssue restrictions in runtime test wrappers #133571 added. The files now match their state before Fix combined ActiveIssue restrictions in runtime test wrappers #133571. The exception is ThisCallTest.cs: its Marshal.GetFunctionPointerForDelegate reverse cases are now skipped on wasm. CoreCLR wasm can't allocate the stub those need at run time, and it isn't reliable on Mono either ([Mono/WASM] Marshal.GetFunctionPointerForDelegate crashes the runtime #104391). Its forward and UnmanagedCallersOnly cases still run there.

Validation

I ran the patched crossgen2 with --generate-portable-callhelpers over the Helix payloads of the three MemberFunction tests. Each one now emits S8ii from Test8ByteHFA.
ThisCall already had S8ii through an UnmanagedFunctionPointer delegate, so only the runtime switch change matters for it.
The new unit test fails without the collector change and passes with it. All 68 WasmArgumentLayoutTests pass. They ran against a browser CoreLib/libs layout staged from the Helix correlation payload.
After the first CI run, all three MemberFunction tests trapped with function signature mismatch in Test8ByteHFAUnmanagedCallersOnly. I reproduced that locally with the Helix payload. The native call_indirect expects (i32, i32, i32) -> void. After the reverse-thunk fix, the regenerated GetSize thunk is void(void* sret, void*, int32_t). The new unit test covers this, and all 69 WasmArgumentLayoutTests pass.
End to end: I did a local browser build (./build.sh -os browser -c Debug -subset clr+libs, which compiles the helpers.cpp change) and built JIT/Directed/callconv with src/tests/build.sh -browser Debug. All four tests exit 100 under node. I reran this after merging main, which brought in [wasm][coreclr] Dispatch R2R-compiled UnmanagedCallersOnly callbacks to their native entrypoint #134355, [wasm][R2R] Fix generic context / async continuation order in Wasm interpreter thunks #134676 and Disable unused ReadyToRun metadata on WebAssembly #134690. All 112 ILCompiler.ReadyToRun.Tests in WasmArgumentLayoutTests pass, and so do the four callconv tests.
After the WasmLowering fix, all 88 WasmArgumentLayoutTests pass. The rebuilt crossgen2 regenerates call helpers for the four callconv tests that are byte-identical to the ones linked in the end-to-end run.

Note on checked-in tables

The checked-in src/coreclr/vm/wasm/{browser,wasi}/callhelpers-*.cpp tables may now be missing framework calli signatures until they are regenerated with generate-coreclr-helpers.sh. A missing entry is not a broken one. They were not regenerated in this PR.

Note

This PR description was generated with AI assistance (GitHub Copilot).

The portable callhelpers generator only collected signatures from P/Invokes, UnmanagedCallersOnly methods and UnmanagedFunctionPointer delegates, so a calli through an unmanaged function pointer whose shape appeared nowhere else had no interp-to-native thunk and asserted in GetCookieForCalliSig. Scan method bodies for unmanaged calli and add their signatures. ComputeCalliSigThunk also rejected IMAGE_CEE_CS_CALLCONV_THISCALL before building a key; on wasm thiscall lowers the same as cdecl. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

azure-pipelines · 2026-09-29T00:44:31Z

Azure Pipelines: Successfully started running 3 pipeline(s). 13 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

dotnet-policy-service · 2026-09-29T00:44:51Z

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

dotnet-policy-service · 2026-09-29T00:53:47Z

Tagging subscribers to 'arch-wasm': @lewing, @pavelsavara
See info in area-owners.md if you want to be subscribed.

…-calli-thunks

Reverts the test disables added in #133571 now that unmanaged calli thunks are generated and thiscall is accepted. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot review overview

🔵 Needs a closer look

The cross-language ABI changes appear coherent, but the native helper was not compiled and the re-enabled tests were not run end to end.

Review effort: Balanced
Findings: None

What changed in this PR

Adds WebAssembly CoreCLR thunk generation for unmanaged calli signatures and permits thiscall, re-enabling affected calling-convention tests.

Changes:

Scans IL for unmanaged calli signatures and emits portable call-helper thunks.
Accepts thiscall during runtime thunk lookup.
Adds generator coverage and restores four browser tests.

File	Description
`src/tests/JIT/Directed/callconv/ThisCall/ThisCallTest.csproj`	Re-enables the browser test.
`src/tests/JIT/Directed/callconv/StdCallMemberFunction/StdCallMemberFunctionTest.csproj`	Restores test-specific browser corerun.
`src/tests/JIT/Directed/callconv/StdCallMemberFunction/StdCallMemberFunctionTest.cs`	Removes the browser skip.
`src/tests/JIT/Directed/callconv/PlatformDefaultMemberFunction/PlatformDefaultMemberFunctionTest.csproj`	Restores test-specific browser corerun.
`src/tests/JIT/Directed/callconv/PlatformDefaultMemberFunction/PlatformDefaultMemberFunctionTest.cs`	Removes the browser skip.
`src/tests/JIT/Directed/callconv/CdeclMemberFunction/CdeclMemberFunctionTest.csproj`	Restores test-specific browser corerun.
`src/tests/JIT/Directed/callconv/CdeclMemberFunction/CdeclMemberFunctionTest.cs`	Removes the browser skip.
`src/coreclr/vm/wasm/helpers.cpp`	Allows `thiscall` thunk lookup.
`src/coreclr/tools/aot/ILCompiler.ReadyToRun/PortableCallHelpers/PInvokeCollector.cs`	Collects unmanaged `calli` signatures from IL.
`src/coreclr/tools/aot/ILCompiler.ReadyToRun.Tests/WasmArgumentLayoutTests.cs`	Tests thunk generation and excluded signatures.

…e thunks The wasm C ABI returns a struct that isn't a single scalar through a hidden leading pointer and no return value. The reverse thunk generator declared such returns as 'void *', so native code calling an [UnmanagedCallersOnly] method returning e.g. a two-float struct trapped with a function signature mismatch (and the interpreter would have written the struct into a 4-byte local). Take the sret pointer as the first parameter and pass it to the interpreter as the return buffer, matching the P/Invoke declaration path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Marshal.GetFunctionPointerForDelegate needs a stub created at run time, which CoreCLR on wasm cannot allocate. Keep the forward and UnmanagedCallersOnly cases running there. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Jan Kotas <jkotas@microsoft.com>

WasmLowering.GetSignature used HasFlag(UnmanagedCallingConvention) to detect unmanaged signatures, but the calling convention is an enum value in the low nibble, not a bitmask, so cdecl/stdcall/thiscall signatures were lowered as managed. Compare the masked value instead and drop the workaround in the calli collector. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…-calli-thunks Resolves conflicts with #134355 in PInvokeTableGenerator.EmitNativeToInterp and WasmArgumentLayoutTests. The R2R dispatch for exported callbacks now forwards the hidden sret pointer, and the rejection of callbacks returning through a hidden buffer is removed because the wrapper now models that ABI. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…-calli-thunks Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

pavelsavara

Do we need to re-generate committed files?

lewing · 2026-09-29T18:56:18Z

Do we need to re-generate committed files?

I don't think so (and double checked browser locally to verify). We do still need to stand up a test that does verify the stubs don't need to be regenerated though.

github-actions Bot added the area-VM-coreclr label Sep 29, 2026

dotnet-policy-service Bot assigned lewing Sep 29, 2026

lewing mentioned this pull request Sep 29, 2026

Fix combined ActiveIssue restrictions in runtime test wrappers #133571

Merged

lewing requested a review from radekdoulik September 29, 2026 00:45

lewing added the arch-wasm WebAssembly architecture label Sep 29, 2026

lewing and others added 2 commits September 28, 2026 20:03

Merge remote-tracking branch 'origin/main' into lewing-wasm-unmanaged…

ec1cde1

…-calli-thunks

Re-enable browser CoreCLR callconv MemberFunction/ThisCall tests

73af018

Reverts the test disables added in #133571 now that unmanaged calli thunks are generated and thiscall is accepted. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

lewing requested a balanced review from Copilot September 29, 2026 02:02

Copilot started reviewing on behalf of lewing September 29, 2026 02:03 View session

Copilot AI reviewed Sep 29, 2026

View reviewed changes

build-analysis Bot mentioned this pull request Sep 29, 2026

Async2TaskAdapters.TestValueTaskOfTaskValueVersusAsync fails with SanityCheck() in Wasm R2R test #133953

Open

lewing and others added 2 commits September 28, 2026 23:19

jkotas reviewed Sep 29, 2026

View reviewed changes

Comment thread src/tests/JIT/Directed/callconv/ThisCall/ThisCallTest.cs Outdated

build-analysis Bot mentioned this pull request Sep 29, 2026

[wasm] OpenQA.Selenium.WebDriverTimeoutException: timeout: Timed out receiving message from renderer #117486

Open

Update src/tests/JIT/Directed/callconv/ThisCall/ThisCallTest.cs

9930ff3

Co-authored-by: Jan Kotas <jkotas@microsoft.com>

radekdoulik approved these changes Sep 29, 2026

View reviewed changes

MichalStrehovsky reviewed Sep 29, 2026

View reviewed changes

Comment thread src/coreclr/tools/aot/ILCompiler.ReadyToRun/PortableCallHelpers/PInvokeCollector.cs Outdated

lewing and others added 3 commits September 29, 2026 11:04

Merge remote-tracking branch 'origin/main' into lewing-wasm-unmanaged…

b32875f

…-calli-thunks Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

lewing requested a review from pavelsavara September 29, 2026 17:08

pavelsavara approved these changes Sep 29, 2026

View reviewed changes

lewing enabled auto-merge (squash) September 29, 2026 18:20

lewing merged commit d0b3d89 into main Sep 29, 2026
132 of 134 checks passed

lewing deleted the lewing-wasm-unmanaged-calli-thunks branch September 29, 2026 19:43

build-analysis Bot mentioned this pull request Sep 29, 2026

System.IO.FileSystem.Tests.WorkItemExecution CI timeout on net9.0-osx-Debug-x64-Mono_Minijit_Debug-OSX.1200.Amd64.Open #101423

Open

dotnet-maestro Bot mentioned this pull request Sep 30, 2026

[main] Source code updates from dotnet/runtime dotnet/dotnet#9845

Merged

dotnet-milestone-bot Bot added this to the 12.0-preview1 milestone Sep 30, 2026

Navigation Menu

[wasm] Generate thunks for unmanaged calli signatures and allow thiscall - #134826

[wasm] Generate thunks for unmanaged calli signatures and allow thiscall#134826
lewing merged 9 commits into
mainfrom
lewing-wasm-unmanaged-calli-thunks

lewing commented Sep 29, 2026 •

edited

Loading

Uh oh!

azure-pipelines Bot commented Sep 29, 2026

Uh oh!

dotnet-policy-service Bot commented Sep 29, 2026

Uh oh!

dotnet-policy-service Bot commented Sep 29, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

pavelsavara left a comment

Uh oh!

lewing commented Sep 29, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

lewing commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Validation

Note on checked-in tables

Uh oh!

azure-pipelines Bot commented Sep 29, 2026

Uh oh!

dotnet-policy-service Bot commented Sep 29, 2026

Uh oh!

dotnet-policy-service Bot commented Sep 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Uh oh!

Uh oh!

pavelsavara left a comment

Choose a reason for hiding this comment

Uh oh!

lewing commented Sep 29, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

lewing commented Sep 29, 2026 •

edited

Loading