Skip to content

Make tool-command resolution and invocation AOT-able - #54810

Merged
baronfel merged 11 commits into
mainfrom
baronfel/tool-resolution-aot
Jun 18, 2026
Merged

baronfel merged 11 commits into
mainfrom
baronfel/tool-resolution-aot

Conversation

@baronfel

@baronfel baronfel commented Jun 16, 2026 •

Copy link
Copy Markdown
Member

Pull external/tool-command resolution and invocation into the NativeAOT bridge (NativeEntryPoint) so commands like global/local tools, PATH commands, and app-base commands resolve and run out-of-process from the AOT fast path, while keeping the managed implementations as the fallback.

Resolution and invocation:

AOT compatibility:

Telemetry:

  • Unify activity/span names across the managed and AOT resolution paths so traces from each host can be compared directly.
  • Resolution telemetry records command, resolution status (found/notfound), and resolver-specific tags; environment variables and argument lists are intentionally not tagged (secrets / unbounded size).
  • Add eng/gather-otel.ps1 and eng/gather-otel.sh helper scripts to capture OpenTelemetry data from a run for validating that the AOT and managed paths emit equivalent telemetry.

Tests and docs:

  • Add NativeEntryPoint tests for unresolved external command and resolvable PATH command paths.
  • Document the external-command resolution path and updated dispatch flow in DESIGN.md.

Performance:

Measured with eng/gather-otel.ps1 (5 managed + 5 AOT runs each) on a Debug build, end-to-end wall-clock per invocation pulled from the dashboard telemetry. Two tools that resolve via different resolvers:

dotnet dev-certs https — bundled DotnetTools tool (resolves at the 2nd resolver, DotnetToolsCommandResolver):

mean median min max
managed 701.4 ms 685 ms 648 ms 815 ms
AOT 219.2 ms 220 ms 215 ms 223 ms
`dotnet dev-certs https` / managed trace image
`dotnet dev-certs https` / NAOT trace image
→ −482 ms, 3.2x faster, 68.7% lower latency.

dotnet ef — PATH / global tool (resolves at the last resolver; the AOT chain has one fewer resolver because ProjectToolsCommandResolver is gated out under !CLI_AOT):

mean median min max
managed 703.4 ms 689 ms 661 ms 768 ms
AOT 199.0 ms 197 ms 195 ms 205 ms
`dotnet ef` / managed trace image
`dotnet ef` / NAOT trace image

→ −504 ms, 3.53x faster, 71.7% lower latency.

The win comes from not booting CoreCLR and re-running the managed CLI a second time (~600-690 ms of the coreclr-invocation span); the actual tool launch (~85-135 ms, the execute-extensible-command span) is shared and identical on both paths.

Binary size impact

NativeAOT-published dotnet-aot (win-x64, Release), measured with sizoscope-cli by diffing the .mstat files of main before this PR's squash-merge versus the full branch.

Metric Pre-PR main This PR Delta
Native dotnet-aot.dll (on disk) 15,368,192 B 17,664,000 B +2.19 MB (+14.9%)
sizoscope accounted (managed, net) — — +1.6 MB

Of the ~2.19 MB on-disk growth, sizoscope attributes ~1.6 MB to managed metadata/code; the remaining ~0.6 MB is native codegen plus alignment/padding.

Top net per-assembly growth drivers:

Assembly Net delta Why
System.Private.CoreLib +687 kB Larger BCL surface made reachable by the new resolution + invocation + telemetry code
System.Linq +148 kB More LINQ shapes instantiated by resolver / telemetry enumeration
dotnet-aot +123 kB This PR's own resolver / invocation / telemetry code
NuGet.Configuration +99 kB Newly reachable — tool/config resolution reads NuGet config
System.Text.Json +94 kB Tool manifest / config deserialization now reachable in AOT
NuGet.Frameworks +75 kB Newly reachable — tool resolution parses framework (tfm) folder names
System.ComponentModel.TypeConverter +40 kB Config/type conversion reachable via NuGet config
System.Private.Xml.Linq +34 kB XML config parsing now in the closure
System.Security.Cryptography +16 kB Telemetry property hashing (Sha256Hasher)
System.Private.Xml +15 kB XML config parsing
Microsoft.DotNet.Cli.Utils +13 kB Command-spec / forwarding + telemetry helpers now in the AOT closure

The growth is expected and intentional: this PR roots the full tool / external-command resolution + invocation chain plus the classic parser/resolution telemetry in the native image, pulling their transitive NuGet/JSON/XML/crypto dependencies into the AOT closure. (The localized *.resources buckets show up as near-equal +/− pairs that re-bucket between builds and net to ~0; sizoscope's accounted total already excludes them.)

Copilot AI review requested due to automatic review settings June 16, 2026 22:18
@baronfel
baronfel requested review from a team as code owners June 16, 2026 22:18
@baronfel baronfel added the Area-dotnet AOT Items that are part of the dotnet CLI AOT-ification effort label Jun 16, 2026
@baronfel
baronfel requested review from JeremyKuhne and NikolaMilosavljevic and removed request for a team and Copilot June 16, 2026 22:18
@Nigusu-Allehu
Nigusu-Allehu self-requested a review June 16, 2026 22:20
Copilot AI review requested due to automatic review settings June 17, 2026 14:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR moves external/tool-command resolution and out-of-process invocation into the NativeAOT bridge (NativeEntryPoint), so AOT can handle more dotnet <cmd> scenarios (tools/PATH/app-base) without hosting the managed CLI, while explicitly gating MSBuild/NuGet object-model paths behind dynamic-code checks and #if !CLI_AOT.

Changes:

  • Add an AOT external-command path (TryInvokeExternalCommand) that resolves via an AOT-safe resolver set and executes out-of-process, otherwise deferring to managed CLI.
  • Thread an sdkRoot through command resolver policy APIs and gate MSBuild/NuGet-based resolution and in-proc MSBuild usage behind AOT/dynamic-code checks.
  • Add/adjust telemetry tags, tests for the new AOT behavior, and update AOT design/docs.

Reviewed changes

Copilot reviewed 39 out of 39 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
test/TemplateEngine/Microsoft.TemplateEngine.Authoring.Tasks.IntegrationTests/LocalizeTemplateTests.cs Update assertions to MSTest’s Assert.HasCount for localized output counts.
test/TemplateEngine/Microsoft.TemplateEngine.Authoring.Tasks.IntegrationTests/CommandResultAssertions.MSTest.cs Align assertions with MSTest equivalents for shared command-result helpers.
test/dotnet.Tests/CommandObjectTests.cs Update ICommandResolverPolicy implementation to accept sdkRoot.
test/dotnet-aot.Tests/NativeEntryPointTests.cs Add tests for AOT external-command fallback and PATH-resolved out-of-proc execution.
src/Microsoft.DotNet.ProjectTools/VirtualProjectBuilder.cs Gate MSBuild usings and annotate MSBuild object-model paths as dynamic-code-required.
src/Cli/Microsoft.DotNet.InternalAbstractions/Properties/Properties.cs Add InternalsVisibleTo for dotnet-aot and dotnet-aot.Tests.
src/Cli/Microsoft.DotNet.Cli.Utils/MSBuildForwardingAppWithoutLogging.cs Guard in-proc MSBuild execution behind RuntimeFeature.IsDynamicCodeSupported and annotate.
src/Cli/dotnet/ReleasePropertyProjectLocator.cs Annotate MSBuild object-model entry points with [RequiresDynamicCode].
src/Cli/dotnet/Program.cs Add activity tagging for external-command lookup.
src/Cli/dotnet/Parser.cs Gate managed action wiring on RuntimeFeature.IsDynamicCodeSupported; annotate managed wiring as dynamic-code-required.
src/Cli/dotnet/Extensions/ParseResultExtensions.cs Duplicate entry-point-path detection to avoid MSBuild dependencies on the AOT path; move CanBeInvoked.
src/Cli/dotnet/Commands/Test/MTP/SolutionAndProjectUtility.cs Annotate MSBuild object-model usage paths as dynamic-code-required.
src/Cli/dotnet/Commands/Test/MTP/MSBuildUtility.cs Annotate MSBuild object-model usage paths as dynamic-code-required.
src/Cli/dotnet/Commands/Test/MTP/MSBuildHandler.cs Annotate handler as dynamic-code-required.
src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs Guard MSBuild handler usage behind RuntimeFeature.IsDynamicCodeSupported.
src/Cli/dotnet/Commands/Run/VirtualProjectBuildingCommand.cs Gate MSBuild-based execution under #if !CLI_AOT and annotate dynamic-code-required helpers.
src/Cli/dotnet/Commands/Run/RunProperties.cs Annotate MSBuild object-model usage path as dynamic-code-required.
src/Cli/dotnet/Commands/Run/RunCommandParser.cs Annotate MSBuild-dependent configuration as dynamic-code-required.
src/Cli/dotnet/Commands/Run/RunCommand.cs Annotate command as dynamic-code-required; whitespace/comment cleanup.
src/Cli/dotnet/Commands/Run/Api/RunApiCommandParser.cs Annotate MSBuild-dependent configuration as dynamic-code-required.
src/Cli/dotnet/Commands/Run/Api/RunApiCommand.cs Annotate MSBuild-dependent command and input execution paths as dynamic-code-required.
src/Cli/dotnet/Commands/Publish/PublishCommandParser.cs Annotate MSBuild-dependent configuration as dynamic-code-required.
src/Cli/dotnet/Commands/Publish/PublishCommand.cs Annotate command as dynamic-code-required.
src/Cli/dotnet/Commands/Project/ProjectCommandParser.cs Annotate MSBuild-dependent configuration as dynamic-code-required.
src/Cli/dotnet/Commands/Project/Convert/ProjectConvertCommand.cs Annotate command as dynamic-code-required.
src/Cli/dotnet/Commands/Pack/PackCommandParser.cs Annotate MSBuild-dependent configuration as dynamic-code-required.
src/Cli/dotnet/Commands/Pack/PackCommand.cs Annotate command as dynamic-code-required.
src/Cli/dotnet/Commands/NuGet/NuGetVirtualProjectBuilder.cs Add suppressions for calling dynamic-code-required APIs via NuGet-owned interfaces.
src/Cli/dotnet/Commands/DotNetCommandFactory.cs Guard MSBuild argument analysis for file-based apps behind RuntimeFeature.IsDynamicCodeSupported.
src/Cli/dotnet/CommandFactory/CommandSpec.cs Gate project-based environment variable extraction behind #if !CLI_AOT.
src/Cli/dotnet/CommandFactory/CommandResolver.cs Add sdkRoot threading and nullable annotations for resolver inputs.
src/Cli/dotnet/CommandFactory/CommandResolution/ICommandResolverPolicy.cs Add sdkRoot parameter to resolver policy API and enable nullable annotations.
src/Cli/dotnet/CommandFactory/CommandResolution/DotnetToolsCommandResolver.cs Add ForSdkRoot factory and resolution activity tags for DotnetTools path.
src/Cli/dotnet/CommandFactory/CommandResolution/DefaultCommandResolverPolicy.cs Thread sdkRoot; exclude project-tools resolver under CLI_AOT.
src/Cli/dotnet/CommandFactory/CommandResolution/CompositeCommandResolver.cs Add per-resolver resolution activities and tags.
src/Cli/dotnet-aot/NativeEntryPoint.cs Add AOT external-command resolution+invocation path and switch dispatch to CanBeInvoked.
src/Cli/dotnet-aot/DESIGN.md Document the new external-command path and updated dispatch/sequence flow.
src/Cli/dotnet-aot/AotSourceFiles.props Include command-resolution/tool-resolution sources in the AOT build.
documentation/general/dotnet-run-file.md Update shebang guidance/examples to prefer /usr/bin/env dotnet.

Comment thread src/Cli/dotnet-aot/NativeEntryPoint.cs
Comment thread src/Cli/dotnet-aot/NativeEntryPoint.cs Outdated
Comment thread src/Cli/dotnet/CommandFactory/CommandResolver.cs Outdated
Comment thread src/Cli/dotnet/CommandFactory/CommandResolver.cs Outdated
Comment thread src/Cli/dotnet/Extensions/ParseResultExtensions.cs Outdated
Comment thread src/Cli/dotnet-aot/NativeEntryPoint.cs
baronfel and others added 7 commits June 18, 2026 11:07
Pull external/tool-command resolution and invocation into the NativeAOT
bridge (NativeEntryPoint) so commands like global/local tools, PATH
commands, and app-base commands resolve and run out-of-process from the
AOT fast path, while keeping the managed implementations as the fallback.

Resolution and invocation:
- Add TryInvokeExternalCommand to NativeEntryPoint: resolve via the AOT-safe
  resolver set and invoke out-of-process; defer file-based apps, legacy
  project tools, unresolved commands, and any resolution error to the
  managed CLI. Switch the dispatch to ParseResult.CanBeInvoked().
- Thread an sdkRoot through CommandResolver/ICommandResolverPolicy/
  DefaultCommandResolverPolicy and add DotnetToolsCommandResolver.ForSdkRoot,
  since the AOT app's AppContext.BaseDirectory is the dotnet root rather than
  the per-SDK version path.
- Gate ProjectToolsCommandResolver and PackagedCommandSpecFactoryWithCliRuntime
  (MSBuild/NuGet-based, AOT-hostile) out of the resolver set under !CLI_AOT.
- Add resolution telemetry tags to CompositeCommandResolver and the tool
  resolvers.

AOT compatibility:
- Annotate MSBuild Object Model code paths with [RequiresDynamicCode] and
  guard in-proc MSBuild usage with RuntimeFeature.IsDynamicCodeSupported
  (RunCommand, RunProperties, VirtualProjectBuilder, DotNetCommandFactory,
  MSBuildForwardingAppWithoutLogging, Pack/Publish/Project/Run-Api/Test MTP).
- Gate MSBuild usings and IProject usage out of the AOT build; duplicate
  IsValidEntryPointPath into ParseResultExtensions to avoid pulling MSBuild
  onto the AOT codepath.
- Expose IFileSystem internals to dotnet-aot via InternalsVisibleTo.
- Add the AOT-safe CommandFactory/tool-resolution sources to AotSourceFiles.props.

Tests and docs:
- Add NativeEntryPoint tests for unresolved external command and resolvable
  PATH command paths.
- Document the external-command resolution path and updated dispatch flow in
  DESIGN.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
- CommandResolver.TryResolveCommandSpec: both overloads can return null, so
  mark the return type CommandSpec? (nullable is enabled in this file).
- CompositeCommandResolver: drop the lookup.env and lookup.args activity tags
  since they can carry secrets (tokens/connection strings) and be very large.
  (The per-miss Error status was already replaced with a lookup.status tag.)
- ParseResultExtensions.IsValidEntryPointPath: make private; it is only a
  helper for GetFileBasedAppEntryPointToken and need not expand the public
  surface area.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When the AOT bubble resolves and invokes an external command (e.g. a global/
local tool like `dotnet dev-certs`), the root activity was left named just
`dotnet` because GetCommandName() only walks parser-matched commands and the
external token is unrecognized. The managed fallback path instead labels the
root span `dotnet <command>` via its best-effort `dotnet {args[0]}` heuristic.

Once TryInvokeExternalCommand commits to handling the command in the bubble,
set the root span DisplayName and command.name tag to `dotnet <subcommand>`
(using the parsed DotnetSubCommand token), matching the managed path's telemetry.

Verified end-to-end via eng/gather-otel.ps1 against the Aspire dashboard: the
AOT root span now reads `dotnet dev-certs` with a matching command.name tag,
while keeping the lean 7-span structure and ~3x warm speedup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
baronfel and others added 4 commits June 18, 2026 11:07
The root AOT span was renamed from 'native-entrypoint' to 'main' in the
telemetry span unification, but ExecuteCore_AotFastPath_CreatesMainActivity
still looked up the activity by the old OperationName, so it found null.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…l --local, and tool search (#54827)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When the NativeAOT bridge handles a command in-process (built-in invocation or external tool resolution+invocation), it now emits the same `toplevelparser/command` and `commandresolution/commandresolved` events the managed CLI sends, instead of dropping them. Events are only emitted on paths that do NOT fall back to the managed host, so they are never double-counted.

- Subscribe TelemetryEventEntry to the AOT TelemetryClient once per process and set the hashing TelemetryFilter (mirrors Program.cs). This alone un-drops the commandresolution/commandresolved event CompositeCommandResolver already raises during external resolution.

- Fire toplevelparser/command via a best-effort SendAotParserTelemetry helper at the three in-process handling points (built-in success, built-in terminal error, external TryInvokeExternalCommand==true); never on fallback paths.

- Move IsDotnetBuiltInCommand out of the #if !CLI_AOT block (it only uses AOT-safe APIs) and source-link TelemetryFilter + its parse-result log rules into dotnet-aot.

- Add tests asserting the events fire for a resolved external command under AOT and are not emitted when deferring to the managed host.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move the `toplevelparser/command` send for externally-resolved tools from after invocation to right after resolution succeeds (the no-fallback commit point) and before `command.Execute()` spawns the external process. This gives the async telemetry uploader the child's entire lifetime to flush instead of racing process exit.

The built-in invocation path keeps firing after `Parser.Invoke` returns: a managed-only command throws CommandNotAvailableInAotException synchronously during invoke and falls back to the managed host, so pre-firing there would double-count the event.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@baronfel
baronfel force-pushed the baronfel/tool-resolution-aot branch from da15d2b to dfbabbf Compare June 18, 2026 16:07
@baronfel
baronfel merged commit 16a0a14 into main Jun 18, 2026
26 checks passed
@baronfel
baronfel deleted the baronfel/tool-resolution-aot branch June 18, 2026 18:29
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 11.0-preview6 milestone Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area-dotnet AOT Items that are part of the dotnet CLI AOT-ification effort

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants