Enable ComWrappers API on all platforms

[sdmaclea]

Our debug interfaces uses COM interfaces.

To properly support these on all platforms it would be ideal to have the underlying wrappers supported.

/cc @AaronRobinsonMSFT @jkoritzinsky

[leculver]

I don't object to enabling COM on non-Windows platforms. As a slight counterpoint though the diagnostics interfaces are just plain old C++ interfaces with Add/Release mechanics. IE not requiring other COM goo like activation.

For example, ClrMD solved this issue without needing COM interop enabled by using well-defined vtable layout and calling addref/release. The COMInterop part of the library defines our own CallableComWrapper which handles addref/release. To use this, we define the vtable layout for an interface, then simply use delegates to call them. It's a lot of typing to build the VTable layout, but it's fairly painless to use once they are defined.

I would ask this question in the opposite direction (not specifically to you Steve but to the diagnostics team/@tommcdon): It's 2020, why is the CLR diagnostics layer still built on COM? Why don't we invest into converting the DBI/ICorDebug into a C# based NuGet package? The benefits are:

1. We can move faster. COM interfaces have a ton of inertia, and it takes forever to code a new feature in the C++ DBI codebase. A C# codebase could move much faster.
2. No need for COM interop to use them.
3. More cross-plat friendly.

Just a few thoughts here. As I said, I don't object if folks really want/need COM turned on for other platforms.

[sdmaclea]

plain old C++ interfaces with Add/Release mechanics.

As far as I was concerned the new COM wrapper API was basically that + IUnknown identity semantics.

We tried to leverage the ClrMD wrappers, but I am told the lack of compliant identity & ICastable semantics was causing issues for us.

I would ask this question in the opposite direction (not specifically to you Steve but to the diagnostics team/@tommcdon): It's 2020, why is the CLR diagnostics layer still built on COM? Why don't we invest into converting the DBI/ICorDebug into a C# based NuGet package?

/cc @dotnet/dotnet-diag

[AaronRobinsonMSFT]

@leculver The ComWrappers API might imply all of COM, but that isn't accurate. The ComWrappers is designed to support the IUnknown ABI with no additional requirements to support anything else related to COM (i.e. apartments, aggregation, etc).

just plain old C++ interfaces with Add/Release mechanics

That's not quite accurate. There is nothing "plain old" about C++ type layout. The implementation is really the IUnknown ABI. C++ has no standard ABI definition here which is why the API is called ComWrappers, but as mentioned could have more accurately been called "IUnknownWrappers".

I don't think anyone wants to support COM as it is on Windows, but the IUnknown contract is very appealing to many in the community for various interop related scenarios.

[leculver]

We tried to leverage the ClrMD wrappers, but I am told the lack of compliant identity & ICastable semantics was causing issues for us.

Ahh yeah, that would cause issues. Good to know!

[AaronRobinsonMSFT]

Speaking of ICastable - #36654. Unsure if ClrMD wrappers could add support.

[noahfalk]

I would ask this question in the opposite direction (not specifically to you Steve but to the diagnostics team/@tommcdon): It's 2020, why is the CLR diagnostics layer still built on COM? Why don't we invest into converting the DBI/ICorDebug into a C# based NuGet package?

There are also native C++ consumers of the COM API (VS is a big one, SOS is another, and then we've got 3rd party tools). If we rewrote all of mscordbi in C# either we have to implement COM-interop (CCW this time) to preserve our public native COM interfaces for existing tools OR we force all tools to rewrite against a new interface. FWIW we have made slow movements towards C# over the years and I expect we'll keep doing it, but the engineering costs to shift 20 years of investment in C++ are quite sizable.

[leculver]

(@sdmaclea I apologize for derailing this issue, I'd be happy to take further replies offline or move them into a new issue.)

Speaking of ICastable - #36654. Unsure if ClrMD wrappers could add support.

That proposal is interesting! Most of the COM wrappers in ClrMD are meant for internal use, they are only exposed as public for a "last resort" when you need to do something directly with ISOSDac or similar interfaces. (Usually it's just me trying to build direct repros for dac issues and not having to special compile ClrMD...)

If something like that ICastableObject design were implemented though I'd likely pick it up in ClrMD, assuming I could continue to support desktop CLR at the same time without needing two separate wrappers.

There are also native C++ consumers of the COM API (VS is a big one, SOS is another, and then we've got 3rd party tools). If we rewrote all of mscordbi in C# either we have to implement COM-interop (CCW this time) to preserve our public native COM interfaces for existing tools OR we force all tools to rewrite against a new interface.

Yep, I understand. I've worked in the DBI codebase and I've dug through VS's debugger implementation several times, so I know how big of a project it would be. =) I won't wade too deeply into design, here but you are right that we'd have to come up with a migration path for existing tools, but we've have built bigger projects before.

I think my more concrete feedback here is that I find it interesting that we haven't moved forward ICorDebug very much in .Net Core when I look at the file history of something like cordebug.idl. If you asked my 5-6 years ago what the future of ClrMD was I would have said "non-existent, because the functionality would mostly be in ICorDebug", but the debugger API seems stagnant.

If I could summarize my thoughts a bit differently: Steve filing this issue made me start thinking about how much value we get out of ICorDebug versus how much effort it is to maintain it and the inertia the com-based codebase generates. My feedback here is "let's dream a bit bigger here instead of just turning the crank and limping along". Moving ICorDebug or some subset of it to C# would be one way to approach that...it's not the end goal.

[noahfalk]

My feedback here is "let's dream a bit bigger here instead of just turning the crank and limping along"

I don't view the lack of progress around ICorDebug as a lack of dreaming big (admitedly I am surely biased - alternate opinions are welcome). I think it is the opposite. The diagnostics team has been continually broadening our ownership and investments so that it includes a wide set of diagnostic APIs, instrumentation, infrastructure and tools of which ICorDebug is just one small part. ICorDebug hasn't changed much because we looked at all the different options where we could deploy our limited time/efforts and decided that we had many opportunities more valuable and more urgent than ICorDebug investments.