Expose project evaluation metrics via System.Diagnostics.Metrics

[OvesN]

Fixes #14419

Context

Project evaluations initiated directly through the MSBuild object model are not visible to the binary logger attached to a later build. This includes hidden evaluations outside a build submission such as loading a Project, creating a ProjectInstance, explicit reevaluations, and project-graph construction.

Changes Made

• Add the process-local Microsoft.Build meter with msbuild.project.evaluations, msbuild.project.evaluation.duration, and msbuild.project.evaluation.pass.duration instruments.
• Record every evaluator invocation separately, including partial-stage and explicit reevaluations. Build-submission evaluations are tagged build_submission; hidden/object-model and project-graph evaluations are tagged outside_build_submission.
• Record Pass0-Pass5 durations beside the existing MSBuildEventSource start/stop boundaries. The pass tag identifies initial_properties, properties, item_definitions, items, using_tasks, or targets.

Metric Contract

• MSBuild exposes the meter but does not configure an exporter or send measurements. Hosts remain responsible for collection, export, and applicable consent policies.
• Measurements are recorded only when a MeterListener enables the corresponding instrument; without a listener, Enabled checks avoid timing and tag work. Non-critical instrumentation failures disable metrics for the remainder of the process without affecting evaluation.

Possible Follow-up

Bridge these process-local metrics into Microsoft.VisualStudio.Telemetry so evaluation aggregates can populate VS telemetry dashboards. This requires early collection for hidden pre-build evaluations and cross-process aggregation; it is intentionally not part of this PR.

[Copilot]

🟡 Changes recommended

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds process-local System.Diagnostics.Metrics instrumentation to Microsoft.Build so hosts can observe project evaluation counts and durations (including per-pass timings) even for evaluations that occur outside of a build submission (object model loads, reevaluations, project graph).

Changes:

• Introduces Microsoft.Build meter instruments for evaluation count, total duration, and per-pass duration with stage/origin/success tags, plus optional submission-id tagging via MSBUILD_EVALUATION_METRICS_INCLUDE_SUBMISSION_ID.
• Wires evaluator start/stop and Pass0–Pass5 timing to emit metric measurements alongside existing MSBuildEventSource boundaries.
• Updates packaging/deployment artifacts to ship System.Diagnostics.DiagnosticSource.dll and adds binding redirects for desktop MSBuild.

File summaries

File	Description
src/Package/MSBuild.VSSetup/files.swr	Adds System.Diagnostics.DiagnosticSource.dll to VS setup payload (and NGEN metadata).
src/MSBuild/app.config	Adds binding redirect for System.Diagnostics.DiagnosticSource for desktop MSBuild.
src/MSBuild/app.amd64.config	Adds binding redirect + codeBase for System.Diagnostics.DiagnosticSource for amd64 MSBuild.
src/Build/TelemetryInfra/EvaluationMetrics.cs	New metrics helper encapsulating meter/instruments, tags, and opt-in submission-id behavior.
src/Build/Microsoft.Build.csproj	References System.Diagnostics.DiagnosticSource and compiles EvaluationMetrics.cs.
src/Build/Evaluation/Evaluator.cs	Emits evaluation + per-pass metric timings around evaluator execution.
src/Build/BackEnd/BuildManager/BuildManager.cs	Uses a process-wide submission id sequence when submission-id correlation is enabled.
src/Build.UnitTests/Telemetry/EvaluationMetrics_Tests.cs	Adds unit tests validating tags, pass alignment, opt-in submission id, and resilience to listener failures.
scripts/Deploy-MSBuild.ps1	Ensures System.Diagnostics.DiagnosticSource.dll is deployed in desktop bootstrap scenarios.

Review details

• Files reviewed: 9/9 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[ViktorHofer]

@baronfel is the use of DiagnosticSource appropriate here? My telemetry knowledge is limited but I was under the impression that we use different mechanisms intentionally.

[baronfel]

We have a few different mechanisms that we need to reconcile:

• EventSource - available today, collection can happen via Windows ETW or dotnet-trace, other EventPipe-based consumers.
  • fine-grained, but requires defining specific contracts
  • isn't inherently scoped to a particular build - will make analysis of server, where we have multiple builds, difficulty
  • may be crank-compatible?
• Build-time telemetry
  • custom handling, emits via the logging service to any configured logger that is listening for telemetry
  • this is scoped to a particular build, now also contains Target/Task statistics
  • pretty coarse, does not integrate well with any existing telemetry collection tools (out of the box)
  • when part of the dotnet CLI, the CLI will emit Activities that contain our telemetry events
• evaluation profiling
  • custom, ends up in ProjectEvaluation ILogger events if the feature is enabled
• this new Metrics-based telemetry
  • uses System.Diagnostics.Metrics, would integrate into OTel-based collection or is visible by dotnet-trace
  • may be automatically crank-compatible, I think one of their happy paths is System.Diagnostics.Metrics-based collection
  • duplicates the existing eval EventSource tracing - could they be unified?
  • has a similar problem to the existing EventSource data - it doesn't identify/track unique build Ids and so will be more difficult to collect in a server-based world

Overall, I think we need to be creating telemetry that

• is easy to get from Crank, as we continue to lean into it
• ideally also works with Perf DDRITs (which I believe ETW can easily today?)
• works in both one-shot AND long-lived scenarios
• is pay-to-play - only causing work if enabled

For System.Diagnostics.Metrics today, I think we need to make sure VS won't yell at us about dll loads if we start including this.

[baronfel]

Left a big long comment and forgot to actually hit 'review'. I'm not going to block this, but I do have some concerns.

[AlesProkop]

Following the telemetry architecture discussion in this PR, I updated the implementation so the existing evaluation EventSource and the new Metrics instruments share a single evaluation lifecycle.

• EvaluationInstrumentation now owns total and pass start/completion, preserving the existing Microsoft-Build EventSource event IDs, payloads, and Pass0-Pass5 boundaries.
• Metrics use the same six-pass model; items covers deferred/lazy item realization rather than introducing a separate pass contract.
• Total metrics complete before ProjectEvaluationFinished, while the EventSource stop retains its established ordering and is guaranteed for failed evaluations. Incomplete passes still do not emit stop events.
• Metrics remain best-effort and available on both modern .NET and .NET Framework. Desktop deployment/binding setup and missing-DiagnosticSource resilience are retained.
• Tests now cover partial stages, build versus outside-build origins, failed evaluations, exact-once measurement emission, ordering, EventSource payload compatibility, deferred items, listener failures, and missing desktop dependencies.

The broader questions around correlation for overlapping builds in long-lived processes and Crank time-series/tag handling remain separate follow-ups; no build IDs were added to metric tags.

[ViktorHofer]

Is the newly introduced System.Diagnostics.DiagnosticSource.dll assembly still getting loaded in VS msbuild scenarios? If not, we should revert its addition to the binding redirects.

[AlesProkop]

Is the newly introduced System.Diagnostics.DiagnosticSource.dll assembly still getting loaded in VS msbuild scenarios? If not, we should revert its addition to the binding redirects.

Yes. In .NET Framework/Visual Studio MSBuild, System.Diagnostics.DiagnosticSource.dll is loaded on the first project evaluation.
Avoiding the load all together would require changing the design: either omit Metrics on .NET Framework or move them into an optional, explicitly loaded adapter.

[ViktorHofer]

OK. We should pay special attention to this feedback comment from Chet:

For System.Diagnostics.Metrics today, I think we need to make sure VS won't yell at us about dll loads if we start including this.

That isn't theoretical. We have seen such regressions when we introduced DiagnosticSource in the past IIRC (before my time). @AR-May might have more details. The last person who worked on this was @YuliiaKovalova.