dontlink.app crash with `EntryPointNotFoundException: xamarin_release_managed_ref`

[spouliot]

Steps to Reproduce

1. Build dontlink.csproj for device (arm64)
2. Run on device

Expected Behavior

Should run the tests

Actual Behavior

2021-04-07 10:40:24.537 dont link[3286:3879143] 
Unhandled Exception:
System.EntryPointNotFoundException: xamarin_release_managed_ref
  at Foundation.NSObject.ReleaseManagedRef () [0x0001f] in /Users/poupou/git/xamarin/xamarin-macios/src/Foundation/NSObject2.cs:285 
  at Foundation.NSObject.Dispose (System.Boolean disposing) [0x00025] in /Users/poupou/git/xamarin/xamarin-macios/src/Foundation/NSObject2.cs:772 
  at Foundation.NSObject.Dispose () [0x00000] in /Users/poupou/git/xamarin/xamarin-macios/src/Foundation/NSObject2.cs:155 
  at ObjCRuntime.Runtime.ThreadPoolDispatcher (System.Func`1[TResult] callback) [0x0000f] in /Users/poupou/git/xamarin/xamarin-macios/src/ObjCRuntime/Runtime.cs:303 
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00009] in /Library/Frameworks/Xamarin.iOS.framework/Versions/Current/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1258 
2021-04-07 10:40:24.539 dont link[3286:3879143] Unhandled managed exception: xamarin_release_managed_ref (System.EntryPointNotFoundException)
  at Foundation.NSObject.ReleaseManagedRef () [0x0001f] in /Users/poupou/git/xamarin/xamarin-macios/src/Foundation/NSObject2.cs:285 
  at Foundation.NSObject.Dispose (System.Boolean disposing) [0x00025] in /Users/poupou/git/xamarin/xamarin-macios/src/Foundation/NSObject2.cs:772 
  at Foundation.NSObject.Dispose () [0x00000] in /Users/poupou/git/xamarin/xamarin-macios/src/Foundation/NSObject2.cs:155 
  at ObjCRuntime.Runtime.ThreadPoolDispatcher (System.Func`1[TResult] callback) [0x0000f] in /Users/poupou/git/xamarin/xamarin-macios/src/ObjCRuntime/Runtime.cs:303 
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00009] in /Library/Frameworks/Xamarin.iOS.framework/Versions/Current/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1258 

=================================================================
	Native Crash Reporting
=================================================================
Got a abrt while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

=================================================================
	Native stacktrace:
=================================================================
	0x105daf554 - /private/var/containers/Bundle/Application/08E86444-779F-4F89-BA69-0BF4ABDE7BA6/dont link.app/dont link : 
	0x105da6170 - /private/var/containers/Bundle/Application/08E86444-779F-4F89-BA69-0BF4ABDE7BA6/dont link.app/dont link : 
	0x105daea9c - /private/var/containers/Bundle/Application/08E86444-779F-4F89-BA69-0BF4ABDE7BA6/dont link.app/dont link : 
	0x1f54d7298 - /usr/lib/system/libsystem_platform.dylib : <redacted>
	0x1f54dcb50 - /usr/lib/system/libsystem_pthread.dylib : pthread_kill
	0x1b2e5bb74 - /usr/lib/system/libsystem_c.dylib : abort
	0x105f22134 - /private/var/containers/Bundle/Application/08E86444-779F-4F89-BA69-0BF4ABDE7BA6/dont link.app/dont link : 
	0x105e06164 - /private/var/containers/Bundle/Application/08E86444-779F-4F89-BA69-0BF4ABDE7BA6/dont link.app/dont link : 
	0x105e8d940 - /private/var/containers/Bundle/Application/08E86444-779F-4F89-BA69-0BF4ABDE7BA6/dont link.app/dont link : 
	0x105e857dc - /private/var/containers/Bundle/Application/08E86444-779F-4F89-BA69-0BF4ABDE7BA6/dont link.app/dont link : 
	0x105e82fe8 - /private/var/containers/Bundle/Application/08E86444-779F-4F89-BA69-0BF4ABDE7BA6/dont link.app/dont link : 
	0x105e8df48 - /private/var/containers/Bundle/Application/08E86444-779F-4F89-BA69-0BF4ABDE7BA6/dont link.app/dont link : 
	0x105e8ddcc - /private/var/containers/Bundle/Application/08E86444-779F-4F89-BA69-0BF4ABDE7BA6/dont link.app/dont link : 
	0x1f54dbcb0 - /usr/lib/system/libsystem_pthread.dylib : _pthread_start
	0x1f54e4778 - /usr/lib/system/libsystem_pthread.dylib : thread_start

=================================================================
	Basic Fault Address Reporting
=================================================================
Memory around native instruction pointer (0x1d7982414):0x1d7982404  ff 0f 5f d6 c0 03 5f d6 10 29 80 d2 01 10 00 d4  .._..._..)......
0x1d7982414  03 01 00 54 7f 23 03 d5 fd 7b bf a9 fd 03 00 91  ...T.#...{......
0x1d7982424  ff 71 ff 97 bf 03 00 91 fd 7b c1 a8 ff 0f 5f d6  .q.......{...._.
0x1d7982434  c0 03 5f d6 90 29 80 d2 01 10 00 d4 03 01 00 54  .._..).........T

=================================================================
	Managed Stacktrace:
=================================================================
=================================================================

Environment

main

Example Project (If Possible)

in repo

[spouliot]

c.c. @rolfbjarne since there were recent related changes wrt coreclr integration

[spouliot]

This started with a5f9700

[spouliot]

Crash occurs async, while running tests, in the thread pool.

In a case it seems it happen on two threads.

Thread 11 name:  Thread Pool Worker
Thread 11 Crashed:
0   libsystem_kernel.dylib        	0x00000001d7982414 __pthread_kill + 8
1   libsystem_pthread.dylib       	0x00000001f54dcb50 pthread_kill + 272
2   libsystem_c.dylib             	0x00000001b2e5bb74 abort + 104
3   dont link                     	0x00000001078b453c xamarin_find_protocol_wrapper_type + 57328956 (runtime.m:1278)
4   dont link                     	0x0000000107797a68 mono_invoke_unhandled_exception_hook + 56162920 (exception.c:1299)
5   dont link                     	0x0000000107820008 mono_thread_internal_unhandled_exception + 56721416 (threads.c:6010)
6   dont link                     	0x0000000107817e54 worker_callback + 56688212 (threadpool.c:0)
7   dont link                     	0x00000001078156c8 worker_thread + 56678088 (threadpool-worker-default.c:476)
8   dont link                     	0x0000000107820610 start_wrapper_internal + 56722960 (threads.c:1289)
9   dont link                     	0x0000000107820494 start_wrapper + 56722580 (threads.c:1310)
10  libsystem_pthread.dylib       	0x00000001f54dbcb0 _pthread_start + 320
11  libsystem_pthread.dylib       	0x00000001f54e4778 thread_start + 8

Thread 12 name:  Thread Pool Worker
Thread 12:
0   libsystem_kernel.dylib        	0x00000001d7982840 __semwait_signal_nocancel + 8
1   libsystem_c.dylib             	0x00000001b2dec68c nanosleep$NOCANCEL + 212
2   libsystem_c.dylib             	0x00000001b2e12dbc usleep$NOCANCEL + 68
3   libsystem_c.dylib             	0x00000001b2e5bb7c abort + 112
4   dont link                     	0x00000001078b453c xamarin_find_protocol_wrapper_type + 57328956 (runtime.m:1278)
5   dont link                     	0x0000000107797a68 mono_invoke_unhandled_exception_hook + 56162920 (exception.c:1299)
6   dont link                     	0x0000000107820008 mono_thread_internal_unhandled_exception + 56721416 (threads.c:6010)
7   dont link                     	0x0000000107817e54 worker_callback + 56688212 (threadpool.c:0)
8   dont link                     	0x00000001078156c8 worker_thread + 56678088 (threadpool-worker-default.c:476)
9   dont link                     	0x0000000107820610 start_wrapper_internal + 56722960 (threads.c:1289)
10  dont link                     	0x0000000107820494 start_wrapper + 56722580 (threads.c:1310)
11  libsystem_pthread.dylib       	0x00000001f54dbcb0 _pthread_start + 320
12  libsystem_pthread.dylib       	0x00000001f54e4778 thread_start + 8

In any case we should not end up in xamarin_find_protocol_wrapper_type... something went wrong earlier (maybe the exception, maybe not)

[filipnavara]

This should have been caught by the code collecting __Internal P/Invokes which should prevent it from being linked out from the native code.

[filipnavara]

I tried to run it locally but I am not sure which configuration is broken for you. For "dontlink > iOS Unified 64-bits - device > Debug [dotnet]" I see -u_xamarin_release_managed_ref in the log. However the build fails with

10:53:18.5605460                      Undefined symbols for architecture arm64:
10:53:18.5605530                        "_ReadFile", referenced from:
10:53:18.5605610                            wrapper_managed_to_native_Interop_Kernel32_ReadFile_System_Runtime_InteropServices_SafeHandle_byte__int_int__intptr in System.Reflection.Metadata.dll.o

so I didn't get to test it on device. The later failure could be fixed by dotnet/runtime#50367.

[spouliot]

For "dontlink > iOS Unified 64-bits - device > Debug [dotnet]"

The failure was not in dotnet tests (at least that's not where I found it) but the legacy one.

It's related to changes that affect both dotnet and legacy code. I would have expected such issue to be much more visible (across more tests) and the current fix is likely just hiding the real, root issue. However it's blocking the next d16-10 preview so it might get merged while a more correct/permanent fix is found and applied.

[filipnavara]

The failure was not in dotnet tests (at least that's not where I found it) but the legacy one.

Makes sense. It was crashing for me with unrelated errors so I didn't try it. The legacy code may skip some of the P/Invoke symbol referencing depending on various options.

UPD: At least the ListExportedSymbols tasks seems to be hit:

15:32:44.0954610   Adding native reference to xamarin_release_managed_ref in __Internal because it's referenced by System.Void Foundation.NSObject::xamarin_release_managed_ref(System.IntPtr,System.Boolean) in Xamarin.iOS.dll. (TaskId:178)

[spouliot]

yeah, there's something else in play since gathering the symbols (for icalls or pinvokes) works in all other cases... and for other apps as well. It's likely a very trivial thing that was overlooked (when the change was made and today when trying to figure it out)