Skip to content

Unify the managed and NativeAOT CLI parsers into one shared implementation - #54653

Merged
baronfel merged 8 commits into
mainfrom
baronfel/unify-cli-parser-entrypoints
Jun 11, 2026
Merged

baronfel merged 8 commits into
mainfrom
baronfel/unify-cli-parser-entrypoints

Conversation

@baronfel

@baronfel baronfel commented Jun 9, 2026

Copy link
Copy Markdown
Member

Goal

Unify the two dotnet CLI parser entrypoints — the managed CLI (dotnet.dll) and the NativeAOT bridge (dotnet-aot.dll) — into a single shared Parser implementation, and push as much parsing work as possible into the native ("AOT bubble") path.

Previously src/Cli/dotnet/Parser.cs was split by one giant top-level #if CLI_AOT … #else … #endif that produced two almost-entirely-different Parser classes: the AOT side hand-rolled a minimal command tree (only --version/--info/sln), while the managed side built the full tree. That duplicated all the shared plumbing and meant the AOT entrypoint could not parse or serve help for the real command surface.

Now both binaries build the same full DotNetCommandDefinition tree and share the same parsing, help, version/info, cli-schema, and exception-handling code. The only differences are small, inline #if CLI_AOT regions where a code path genuinely cannot run under AOT yet — and in those cases the command transparently falls back to the managed CLI.

How the AOT fast-path works

When DOTNET_CLI_ENABLEAOT=true, NativeEntryPoint parses with the shared parser and:

  • Commands that can run entirely in AOT (e.g. --version, --info, --cli-schema, --help for every command, and the AOT-capable sln subcommands) execute natively and return.
  • Every other command is wired with a fallback action that throws CommandNotAvailableInAotException; the bridge catches it (and any unexpected parse-time failure) and hosts the managed dotnet.dll to run the command — so behavior is identical, just slower on first use.

Highlights

One shared Parser.cs

  • Collapsed the top-level #if into a single Parser class. Shared members (response-file handling, parser/invocation config, Parse/Invoke/InvokeAsync, GetBuiltInCommand, ExceptionHandler, DotnetHelpBuilder) are written once.
  • Mode-specific wiring lives in small inline #if CLI_AOT blocks (ConfigureManagedActions vs ConfigureAotActions), matching the patterns already used elsewhere in the repo (SolutionCommandParser, Program.cs).

Full command surface parses + helps from AOT

  • The AOT build now constructs the complete DotNetCommandDefinition, so parsing and --help match the managed CLI exactly.
  • --help is served entirely from the native path for every command — including the external-tool commands (msbuild/nuget/vstest/format/fsi). Rather than special-casing help, the forwarding apps themselves were made AOT-friendly so the help writer (DotnetHelpBuilder) has zero conditional compilation.

Forwarding apps made AOT-safe (out-of-process)

  • MSBuildForwardingApp forces out-of-process forwarding under AOT (added forceOutOfProc to MSBuildForwardingAppWithoutLogging) and skips the in-proc engine; the LLM-environment terminal-logger adjustment from CommonRunHelpers.AdjustMSBuildForLLMs now applies in AOT too (only the in-proc GetConsoleLogger is excluded).
  • NuGetCommand always uses the shelling-out NuGetForwardingApp runner under AOT (never the in-proc NuGet.CommandLine.XPlat runner).
  • fsi/format/vstest forwarders (already process-spawn) plus their dependencies were added to the AOT compile set.

Shared option actions

  • --version and --info use the same PrintVersionAction/PrintInfoAction as the managed CLI; --info only #if !CLI_AOT-omits the workload/MSBuild version lines that aren't AOT-compatible yet.
  • --cli-schema runs in AOT via the source-generated JsonSerializerContext. As part of this, CliSchema.HumanizeValue now renders all enum option/argument defaults as their name (previously only VerbosityOptions), which both reads better and is required for source-gen serialization; GetJsonSchema was switched to the AOT-safe JsonTypeInfo overload of GetJsonSchemaAsNode. CliSchema.cs now has no AOT conditional compilation.
  • ExceptionHandler is now shared and invoked from the AOT path for non-fallback exceptions, keeping error reporting at parity with the managed CLI.

Cleanup

  • Removed CommandLineInfo.cs — it was an AOT-only duplicate of the version/info output that became dead once the real PrintVersionAction/PrintInfoAction were made AOT-able.

Validation

  • Managed dotnet.csproj builds clean; behavior is unchanged (logic reorganized, not altered).
  • dotnet-aot.csproj AOT-publishes for win-x64 with 0 ILC/trim warnings (the full command tree + all included forwarders + cli-schema serialization are trim/AOT-clean).
  • All dotnet-aot.Tests pass, including new coverage for dotnet --help, build --help, external-tool help rendering from AOT, --cli-schema JSON output, and the managed-fallback contract.

Note: the managed CliSchemaTests require the full redist muxer layout, which isn't built in a worktree-only checkout, so they were validated indirectly via the AOT full-tree serialization test. They should be re-run in CI.

@baronfel
baronfel requested review from a team, JeremyKuhne and NikolaMilosavljevic June 9, 2026 18:22
@baronfel
baronfel marked this pull request as ready for review June 9, 2026 21:30
@baronfel
baronfel requested a review from a team as a code owner June 9, 2026 21:30
Copilot AI review requested due to automatic review settings June 9, 2026 21:30
@Nigusu-Allehu
Nigusu-Allehu self-requested a review June 9, 2026 21:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR unifies the managed dotnet.dll and NativeAOT bridge dotnet-aot.dll CLI parsing so both entrypoints build the same full DotNetCommandDefinition command tree, enabling consistent parsing/help output and allowing the AOT path to execute a limited set of commands while falling back to the managed CLI for everything else.

Changes:

  • Collapses the previous split managed/AOT parser into one shared Parser with small #if CLI_AOT action-wiring differences.
  • Enables AOT to render --help across the full command surface (including external-tool forwarders) and to emit --cli-schema.
  • Updates AOT tests and AOT build inputs to cover full-tree parsing/help/schema plus managed-fallback behavior; removes the now-dead CommandLineInfo duplicate.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
test/dotnet-aot.Tests/dotnet-aot.Tests.csproj Updates import comment for shared AOT sources.
test/dotnet-aot.Tests/AotParserTests.cs Expands test coverage to full-tree parsing, help rendering, cli-schema, and fallback behavior.
src/Common/EnvironmentVariableNames.cs Adds XML doc for DOTNET_CLI_TELEMETRY_OPTOUT.
src/Cli/Microsoft.DotNet.Cli.Utils/MSBuildForwardingAppWithoutLogging.cs Adds forceOutOfProc support and extensive XML docs around MSBuild forwarding behavior.
src/Cli/Microsoft.DotNet.Cli.Definitions/Commands/DotNetCommandDefinition.cs Moves to a shared root Command definition, adds recursive --help, and adjusts directive handling.
src/Cli/dotnet/ParserOptionActions.cs Shares option actions across managed/AOT, adds AOT-friendly option-presence checks, and AOT-safe --info/--cli-schema behavior.
src/Cli/dotnet/Parser.cs Centralizes parser creation and adds AOT fallback wiring, shared help builder, and shared exception handling.
src/Cli/dotnet/NuGetSignatureVerificationEnabler.cs Excludes MSBuild-specific overload from AOT compilation.
src/Cli/dotnet/Extensions/ParseResultExtensions.cs Moves parsing helpers needed by AOT out of #if !CLI_AOT and refactors shared helpers.
src/Cli/dotnet/Commands/Run/CommonRunHelpers.cs Excludes in-proc MSBuild logger creation from AOT while keeping LLM adjustments shared.
src/Cli/dotnet/Commands/NuGet/NuGetCommand.cs Forces out-of-proc NuGet forwarding under AOT.
src/Cli/dotnet/Commands/MSBuild/MSBuildForwardingApp.cs Forces out-of-proc MSBuild forwarding under AOT and adjusts distributed logger wiring for AOT constraints.
src/Cli/dotnet/CommandLineInfo.cs Removes obsolete AOT-only version/info implementation.
src/Cli/dotnet/CliSchema.cs Switches to AOT-safe schema export overload and improves enum default rendering.
src/Cli/dotnet-aot/NativeEntryPoint.cs Uses shared parser + shared exception handling; adds parse-time fallback safety and structured activities.
src/Cli/dotnet-aot/DESIGN.md Updates design doc to reflect full-tree AOT parsing + fallback model and shared option actions.
src/Cli/dotnet-aot/AotSourceFiles.props Updates linked shared sources and adds forwarding-app sources needed for AOT help.
src/Cli/dn/dn-native-debug.vcxproj Removes now-deleted CommandLineInfo.cs from native debug project items.

Comment thread src/Cli/dotnet/Parser.cs
Comment thread src/Cli/Microsoft.DotNet.Cli.Definitions/Commands/DotNetCommandDefinition.cs Outdated
Comment thread src/Cli/dotnet/Commands/MSBuild/MSBuildForwardingApp.cs
Comment thread src/Cli/dotnet/Commands/MSBuild/MSBuildForwardingApp.cs
@baronfel

Copy link
Copy Markdown
Member Author

I used sizoscope-cli to diff the impact of this PR on the generated NAOT library:

> dnx sizoscope-cli \
    E:\code\dotnet-sdk\artifacts\obj\dotnet-aot\Release\net11.0\win-x64\native\dotnet-aot.mstat \
    E:\code\copilot-worktrees\sdk\baronfel-studious-eureka\artifacts\obj\dotnet-aot\Release\net11.0\win-x64\native\dotnet-aot.mstat \
  --output dotnet-aot-sizoscope-diff.md

dotnet-aot-sizoscope-diff.md

Net result is +781kB, mostly from the Microsoft.DotNet.Cli.Definitions library and embedded resource strings for the many languages we support.

Comment thread src/Cli/dotnet/Commands/MSBuild/MSBuildForwardingApp.cs
Comment thread src/Cli/dotnet/Commands/MSBuild/MSBuildForwardingApp.cs
Comment thread src/Cli/dotnet/Extensions/ParseResultExtensions.cs
baronfel and others added 8 commits June 11, 2026 13:12
… to managed CLI

Extracts the file-based app detection condition from Program.TryRunFileBasedApp
into a shared ParseResultExtensions.GetFileBasedAppEntryPointToken helper (built
on the AOT-safe VirtualProjectBuilder.IsValidEntryPointPath). The managed CLI
reuses it to re-dispatch 'dotnet app.cs' as 'dotnet run --file', and the AOT
NativeEntryPoint uses it to defer such invocations to the managed CLI rather
than printing root usage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…or MSBuild cleanup

- AOT NativeEntryPoint now defers any unrecognized top-level token to the managed
  CLI, not just file-based apps: 'dotnet <external>' (e.g. dotnet ef) and
  'dotnet app.cs' both land on the root's hidden subcommand argument and require
  managed external-command / run-pipeline resolution. Generalized the detection
  into ParseResultExtensions.RequiresManagedCommandResolution.
- Removed an unused using, fixed an unindented comment, filled in an empty XML
  doc summary, and fixed a grammar typo in the MSBuild forwarding apps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NormalizeRootOptions matched the help option with a name-based check
(Option<bool> whose Name equals --help), a leftover from when
DotNetCommandDefinition briefly carried an explicit help option. With
DotNetCommandDefinition back to a RootCommand base, the
System.CommandLine-injected HelpOption's Name is not --help, so the match
failed and the option kept the default S.CL help action. That caused every
command's --help/-h to render via the default builder (wrong description,
default-value display, arg-name display) and broke the help-output snapshot
tests across dotnet.Tests and dotnet-new.IntegrationTests.

Restore the canonical 'option is HelpOption' type check (matching main) so
help is served by DotnetHelpBuilder again.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@baronfel
baronfel force-pushed the baronfel/unify-cli-parser-entrypoints branch from 11891d4 to 41badfe Compare June 11, 2026 18:17
@baronfel

Copy link
Copy Markdown
Member Author

/ba-g test failures aren't related to this change, known flaky tests

@baronfel
baronfel merged commit 2fc1745 into main Jun 11, 2026
25 of 27 checks passed
@baronfel
baronfel deleted the baronfel/unify-cli-parser-entrypoints branch June 11, 2026 21:47
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 11.0-preview6 milestone Jun 12, 2026
NikolaMilosavljevic added a commit to NikolaMilosavljevic/sdk that referenced this pull request Jun 15, 2026
Per review feedback, the unified parser (dotnet#54653) renders the full command tree and help from AOT, so falling back to the managed CLI just to print a missing-command error + usage is unnecessary overhead. Bare 'dotnet sdk' and 'dotnet sln' now render from AOT; only 'sln add' still falls back since it requires MSBuild. Moved HandleMissingCommand out of the !CLI_AOT block so it compiles into AOT.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NikolaMilosavljevic added a commit to NikolaMilosavljevic/sdk that referenced this pull request Jun 16, 2026
Running the AOT CLI tests in CI surfaced real failures from the new tests added in dotnet#54653:

* GetFileBasedAppEntryPointToken pulls in Microsoft.Build, which cannot be linked into a NativeAOT image (ILC reports the method 'will always throw'). Skip the three affected AotParserTests under NativeAOT (guarded by RuntimeFeature.IsDynamicCodeSupported, so the managed test run still exercises them). Tracked by dotnet#54806.

* InvokeCliSchema_RendersSchemaJsonFromAot evaluates a shell-completion default that requires SHELL to be set; the bare AzDO script task does not export it on Linux. Set SHELL on the AOT test step.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NikolaMilosavljevic added a commit to NikolaMilosavljevic/sdk that referenced this pull request Jun 17, 2026
…al, and tool search

Align the AOT-capable `dotnet tool` subcommands with the unified parser model
(PR dotnet#54653) used by the `sln` and `sdk check` commands: link the real command
implementations into the AOT CLI instead of maintaining a separate self-contained
handler, and wire them via an AOT-aware ToolCommandParser.ConfigureCommand.

- ToolCommandParser: under #if CLI_AOT, the local `list`/`uninstall`, `run`,
  and `search` paths call the real commands. The bare `tool` command, the
  `--global`/`--tool-path` variants, and `install`/`update`/`restore`/
  `execute` throw CommandNotAvailableInAotException so NativeEntryPoint falls
  back to the managed CLI.
- NugetToolSearchApiRequest: swap the NuGet.Protocol service-index lookup for an
  HttpClient + System.Text.Json source-gen path under #if CLI_AOT; the rest of the
  search pipeline (STJ deserializer, PrintableTable) is already AOT-friendly.
- CommandFactoryUsingResolver / CommandSpec: guard the resolver-policy overloads
  and the project-based env-var helper under #if !CLI_AOT, keeping only the
  CommandSpec path that `tool run` needs.
- AotSourceFiles.props: link the tool manifest/package infrastructure, the local
  list/uninstall/run/search implementations, and the NuGet search sources.
- Grant InternalsVisibleTo to dotnet-aot(.Tests) from InternalAbstractions for
  IFileSystem.
- AotParserTests: cover parsing of the AOT-capable tool commands and assert the
  managed-only variants fall back via CommandNotAvailableInAotException.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NikolaMilosavljevic added a commit to NikolaMilosavljevic/sdk that referenced this pull request Jun 17, 2026
…al, and tool search

Align the AOT-capable `dotnet tool` subcommands with the unified parser model
(PR dotnet#54653) used by the `sln` and `sdk check` commands: link the real command
implementations into the AOT CLI instead of maintaining a separate self-contained
handler, and wire them via an AOT-aware ToolCommandParser.ConfigureCommand.

- ToolCommandParser: under #if CLI_AOT, the local `list`/`uninstall`, `run`,
  and `search` paths call the real commands. The bare `tool` command, the
  `--global`/`--tool-path` variants, and `install`/`update`/`restore`/
  `execute` throw CommandNotAvailableInAotException so NativeEntryPoint falls
  back to the managed CLI.
- NugetToolSearchApiRequest: swap the NuGet.Protocol service-index lookup for an
  HttpClient + System.Text.Json source-gen path under #if CLI_AOT; the rest of the
  search pipeline (STJ deserializer, PrintableTable) is already AOT-friendly.
- CommandFactoryUsingResolver / CommandSpec: guard the resolver-policy overloads
  and the project-based env-var helper under #if !CLI_AOT, keeping only the
  CommandSpec path that `tool run` needs.
- AotSourceFiles.props: link the tool manifest/package infrastructure, the local
  list/uninstall/run/search implementations, and the NuGet search sources.
- Grant InternalsVisibleTo to dotnet-aot(.Tests) from InternalAbstractions for
  IFileSystem.
- AotParserTests: cover parsing of the AOT-capable tool commands and assert the
  managed-only variants fall back via CommandNotAvailableInAotException.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants