.NET projects must use native MTP for testing

[ptr727]

Migration Guide

[ptr727]

Concrete fleet-hosted breakage found while resyncing ptr727/LanguageTags (fix: ptr727/LanguageTags#325): the hub's own centrally-hosted .github/workflows/validate-task.yml unit-test step still runs dotnet test --collect:"XPlat Code Coverage" --results-directory ./coverage, the VSTest coverlet.collector invocation. Any repo whose test project pulls in Microsoft.Testing.Platform.MSBuild transitively (e.g. xunit.v3 4.0.0+) fails outright on the .NET 10 SDK with:

error : Testing with VSTest target is no longer supported by Microsoft.Testing.Platform on .NET 10 SDK and later. If you use dotnet test, you should opt-in to the new dotnet test experience. For more information, see https://aka.ms/dotnet-test-mtp-error

This is currently blocking dependabot's own nuget-deps bump PRs on repos hitting this (ptr727/LanguageTags#323, #324).

What actually worked in LanguageTags#325, migrating to native MTP mode per the migration guide already linked on this issue:

• Add global.json: {"test":{"runner":"Microsoft.Testing.Platform"}}.
• Drop xunit.runner.visualstudio (VSTest adapter, unneeded under native MTP).
• Swap coverlet.collector for coverlet.MTP (coverlet's own VSTest data collector is silently ignored under MTP, MTP0001) with no PrivateAssets/IncludeAssets restriction, since coverlet.MTP participates in compile-time self-registration codegen and needs to stay on the compile-time closure, unlike the old build-only collector.
• CI's unit-test step becomes dotnet test --coverlet --coverlet-output-format cobertura --results-directory ./coverage in place of --collect:"XPlat Code Coverage" — this still lands Cobertura XML at ./coverage for the existing Codecov upload step unchanged.

Since validate-task.yml is hub-hosted (repos are meant to delete their own carried copy and call this one, per the fleet audit's hub-only finding), the hub's own unit-test step needs this same migration, plus WORKFLOW.md's D1.6 guarantee text (which currently cites dotnet test --collect:"XPlat Code Coverage" as the canonical invocation) needs updating to match. Until that lands, a repo that follows the audit's instruction to delete its local validate-task.yml copy in favor of the hub's reusable one would silently stop reporting coverage (best-effort upload, so it wouldn't fail CI, just go dark).

[ptr727]

Fleet Catalog and Hub Fix Plan

Surveyed all 7 dotnet repos in the registry (Utilities, LanguageTags, PlexCleaner, NxWitness, PhotoCleaner, MediaTools, AudioCleaner) against their live GitHub state today (2026-08-29). This is planning output only, no repo or hub file has been changed yet. Execution happens in a follow-up session, per repo, as its own PR.

Catalog

Repo	xunit.v3	xunit.runner.visualstudio	Coverlet package	global.json MTP opt-in	CI test step source	Status
Utilities	3.2.2 (main)	3.1.5	coverlet.collector 10.0.1	absent	local validate-task.yml	Broken, fix in flight (#447 to develop, #448 to main)
LanguageTags	4.0.0 (develop) / 3.2.2 (main)	absent (develop) / 3.1.5 (main)	coverlet.MTP 10.0.1 (develop)	present (develop)	local validate-task.yml	Migrated on develop (#325), not yet promoted to main
PlexCleaner	3.2.2 (main), bump to 4.0.0 pending	3.1.5, bump to 4.0.0 pending	coverlet.collector 10.0.1	absent	local validate-task.yml	Broken, blocking open dependabot PRs #937/#938
NxWitness	3.2.2, bump to 4.0.0 pending	3.1.5, bump to 4.0.0 pending	coverlet.collector 10.0.1	absent	local validate-task.yml	Broken, blocking open dependabot PRs #545/#546
PhotoCleaner	4.0.0 (merged via dependabot #82)	4.0.0	coverlet.collector 10.0.1	absent, rejected on purpose	hub reusable validate-task.yml, pinned SHA	Broken, worked around with IsTestingPlatformApplication=false to force VSTest
MediaTools	3.2.2	3.1.5	coverlet.collector 6.0.4	absent	local validate-task.yml	Not yet hit, confirmed passing at this pin
AudioCleaner	3.2.2	3.1.5	coverlet.collector 10.0.1	absent	local validate-task.yml	Not yet hit, confirmed passing at this pin

Empirically confirmed (local dotnet test runs against MediaTools and AudioCleaner on .NET SDK 10.0.400): the break is specific to xunit.v3 4.0.0 and later, not merely the presence of Microsoft.Testing.Platform.MSBuild in the dependency graph. Both repos already pull that package transitively at xunit.v3 3.2.2 and both still pass under the old dotnet test --collect:"XPlat Code Coverage" invocation. This narrows and corrects the issue's own background text, which reads as if any MTP-transitive dependency triggers the failure.

How They Diverged

Six of the seven repos carry their own local copy of validate-task.yml rather than calling the hub's reusable workflow, so each one that hit the failure was free to fix it independently, and each picked a different answer:

1. LanguageTags (Exempt operational repos from the baseline App-secret requirement #325, merged to develop): global.json opt-in to Microsoft.Testing.Platform, dropped xunit.runner.visualstudio, swapped coverlet.collector for coverlet.MTP with no PrivateAssets/IncludeAssets restriction, and changed the CI invocation to dotnet test --coverlet --coverlet-output-format cobertura --results-directory ./coverage. Verified locally (310/310 tests, cobertura output produced) before merge.
2. Utilities (Fix hub bugs surfaced by the config-repo convergence #447/Promote the config-repo convergence hub fixes to main (#447) #448, open): the same global.json opt-in, but coverage via Microsoft.Testing.Extensions.CodeCoverage (Microsoft's own native MTP coverage extension) instead of coverlet, with CI invocation dotnet test --coverage --coverage-output-format cobertura --coverage-output coverage.cobertura.xml --results-directory ./coverage.
3. PhotoCleaner (dependabot Develop ruleset strict "branches up to date" flag stalls bot auto-merge when sibling PR lands mid-CI #82, merged): stayed on VSTest by adding <IsTestingPlatformApplication>false</IsTestingPlatformApplication> to the test csproj. Its own OPERATIONS.md runbook states the global.json route was considered and rejected because it "breaks the --collect:\"XPlat Code Coverage\" argument this repo's CI relies on," since that argument lives in the hub's reusable workflow, not in a file PhotoCleaner controls. This is a real fix for one dependency bump but not a durable one, the next xunit.v3/Test.Sdk generation to drop the VSTest bridge entirely will break it again with no local file left to patch.
4. PlexCleaner and NxWitness have not fixed it yet. Both have open, currently-failing dependabot PRs (Stop merge-and-release's Hub Refresh from Force-Discarding Local main #937/Add SITE_AUTH_TOKEN_ID/SITE_AUTH_TOKEN Secret Handoff to Deploy-Site Verify Hook #938 and Take the ruleset bypass list out of the fleet config entirely #545/Queue the agent-isolation rule and the vendored-tooling surface #546) blocked on the identical error, each preceded by an earlier dependabot attempt that was closed and re-rolled without success (Add markdown-exclude-globs Input to validate-task.yml #935/Promote Develop to Main #936 and Delete the stale PhotoCleaner driftNotes #537/configure.sh check mode reports a clean audit over drift it never compares #538).

PhotoCleaner's case is the one that matters most for this issue: it is the only repo of the seven actually consuming the hub's reusable validate-task.yml (confirmed via docs/reusable-workflows.md's rollout tracker, it is the pilot for that migration), and it is also the one repo structurally unable to apply either of the two proven fixes on its own. Every future repo that converges onto the hub's reusable workflow, which is the fleet's stated direction, inherits the same bind unless the hub itself carries a working MTP invocation.

Hub Changes Needed

1. .github/workflows/validate-task.yml, unit-test job (line 282): replace dotnet test --collect:"XPlat Code Coverage" --results-directory ./coverage with the MTP-native form. This requires picking one coverage engine, see the open decision below, both proven forms are one line each:
   • coverlet route: dotnet test --coverlet --coverlet-output-format cobertura --results-directory ./coverage
   • Microsoft.Testing.Extensions.CodeCoverage route: dotnet test --coverage --coverage-output-format cobertura --coverage-output coverage.cobertura.xml --results-directory ./coverage
2. WORKFLOW.md D1.6: update the cited canonical invocation to match, and add the global.json MTP opt-in as a stated prerequisite for a C# repo with a test project, since the hub cannot supply that file itself, it lives in each consumer's own root.
3. A new audit check: a dotnet repo whose test project pulls in Microsoft.Testing.Platform.MSBuild (directly or transitively, in practice xunit.v3 4.0.0+) but carries no global.json test.runner opt-in is exactly the drift this issue exists to prevent finding out about only when dependabot's bump PR goes red. Making this mechanically detectable means the audit catches it ahead of the break instead of after it, which is how PlexCleaner, NxWitness, and Utilities each found out.
4. Sequencing is safe because callers pin the reusable workflow by SHA (per the fleet's SHA-pin convention), so changing the hub's main does not retroactively break any repo still pinned to the current commit. The one exception is PhotoCleaner, which needs its own repo-side migration (the global.json plus whichever coverage package is chosen) landed in the same PR that bumps its pin to the new hub commit, mirroring how every self-migrating repo above paired its own fix with its own workflow edit.

Open Decision

Two structurally different, independently proven fixes exist in the fleet right now (coverlet.MTP in LanguageTags, Microsoft.Testing.Extensions.CodeCoverage in Utilities' in-flight PRs). The hub needs to standardize on one so every repo that migrates lands on the same shape, and so PhotoCleaner has exactly one route once it can no longer defer. Recommendation: coverlet.MTP, since it is the one already fully round-tripped through a merged PR, keeps the same coverage engine and Cobertura output shape the fleet's existing Codecov pipeline already assumes, and needs no change to codecov.yml semantics. This is a call for the maintainer, not something to resolve unilaterally, particularly since the Utilities PRs choosing the other route are open right now.

Recommended Execution Order (Follow-Up Session)

1. Land the hub fix (validate-task.yml plus WORKFLOW.md D1.6) once the coverage-engine decision is made, since it is non-breaking to every current pin.
2. Add the audit check for the global.json prerequisite.
3. Per-repo migration PRs, one per repo, each pairing its own global.json/package/CI change with (where applicable) a pinned-SHA bump to the new hub commit: PlexCleaner and NxWitness first (already broken, blocking real dependabot PRs), then PhotoCleaner (needs the hub fix landed first, it has no local file to patch ahead of that), then MediaTools and AudioCleaner (not yet broken, lower urgency), then LanguageTags' own develop-to-main promotion, and Utilities' own Fix hub bugs surfaced by the config-repo convergence #447/Promote the config-repo convergence hub fixes to main (#447) #448 reconciled with whichever engine the hub adopts.
4. Track the rollout with a checkbox tracker, the same pattern used for the LF and reusable-workflows migrations, since this spans 6-7 independent repo PRs plus the hub PR.
5. Separately from this issue: 6 of 7 dotnet repos still carry a local copy of validate-task.yml rather than the hub's reusable one, against the fleet's stated hub-only direction for that file. Worth its own tracking, not a blocking prerequisite for this fix.

[ptr727]

Hub half implemented in #1107 (feature -> develop), scoped to the two unblocking changes: the reusable validator's unit-test step and WORKFLOW.md D1.6 with its spec/project-types.json mirror. The mechanical audit check from item 3 is deferred to #1108.

Open decision resolved: Microsoft.Testing.Extensions.CodeCoverage, not coverlet.MTP. Maintainer's call, on the reasoning that it is the platform's own direction, and Utilities' in-flight PRs already chose it.

Two corrections to this issue's plan, both found empirically rather than by reading, on SDK 10.0.400 against a constructed two-project solution. Each matters for every per-repo migration PR that follows, so the recommended execution order should carry them.

1. The invocation in this issue's item 1 loses coverage on a multi-test-project solution. --coverage-output coverage.cobertura.xml pins one filename for every test project, so the second silently overwrites the first: both tests pass, exit 0, and one project's coverage is simply absent from the upload. The old --collect form wrote ./coverage/<guid>/coverage.cobertura.xml and could not collide.

2. Dropping that flag alone then uploads nothing at all. The default name it produces, <guid>.cobertura.xml, is one codecov-cli's own file finder does not match, its patterns being *coverage*.* and an exact cobertura.xml. With fail_ci_if_error: false that is silent. Migrate the Hub's Unit-Test Step to Native Microsoft.Testing.Platform #1107 therefore leaves the filename unset and prefixes each report to coverage-<guid>.cobertura.xml, keeping the guid that makes reports unique while restoring a name the finder matches. Verified by running codecov-cli's FileFinder over the tree the step actually produces.

A version floor this issue does not mention. Microsoft.Testing.Extensions.CodeCoverage at 18.0.4, which is what a >= 18.0.0 range resolves to, is built against MTP 1.x and throws a TypeLoadException against the MTP 2.x that xunit.v3 4.0.0 carries: zero tests run, exit 7, and it still writes a well-formed Cobertura file reporting line-rate="1". Only the exit code distinguishes it from a clean run. 18.1.0 is the first MTP 2.x build, and 18.9.0 is the first on MTP 2.3.x, where every test project writes into the one shared results directory the invocation names. D1.6 states 18.9.0 as the floor.

One wording change worth carrying into the per-repo work: the prerequisites are written as binding on a repo whose test project is MTP-based, not on every C# repo. Otherwise all seven dotnet repos fail the csharp.coverage.codecov letter assert the moment the hub change merges, for a transitional state the same change calls legitimate. A repo still on VSTest holding its existing pin is a migration owed, not drift.

Items 3 (#1108), 4, and 5 of the execution order remain open, as do the per-repo migration PRs.