Bootstrap MSBuild unable to build console projects: unable to resolve workload SDKs

[KirillOsenkov]

Prep a bootstrap MSBuild layout using build /p:CreateBootstrap=true
or download the one I used from:
https://kirillosenkovfiles.blob.core.windows.net/kirillosenkovblob/msbuild.17.3.1.zip

Be on an empty Windows machine, such as Windows Sandbox VM

Create and build a new C# console app targeting net472

Expected: builds successfully

Actual:

Build FAILED.

  C:\Users\WDAGUtilityAccount\AppData\Local\Bootstrap\MSBuild\Sdks\Microsoft.NET.Sdk\targets\Mic
rosoft.NET.Sdk.ImportWorkloads.props(14,3): error : Unable to locate the .NET SDK. Check that it is installed and that
the version specified in global.json (if any) matches the installed version.
  C:\Users\WDAGUtilityAccount\AppData\Local\Bootstrap\MSBuild\Sdks\Microsoft.NET.Sdk\targets\Mic
rosoft.NET.Sdk.ImportWorkloads.targets(16,3): error : Unable to locate the .NET SDK. Check that it is installed and tha
t the version specified in global.json (if any) matches the installed version.

The two SDKs it can't resolve:

Resolving SDK 'Microsoft.NET.SDK.WorkloadAutoImportPropsLocator'...
Resolving SDK 'Microsoft.NET.SDK.WorkloadManifestTargetsLocator'...

First off, the error messages are inadequate - they don't mention the SDK reference that failed to resolve.

Both of these are supposed to resolve from:
https://github.com/dotnet/sdk/blob/3dc5e528fffc050cace4cff8bc32954eb33f0455/src/Resolvers/Microsoft.NET.Sdk.WorkloadMSBuildSdkResolver/CachingWorkloadResolver.cs#L121-L136

However on an empty machine with just the xcopied MSBuild something goes awry apparently and it fails with the obscure messages above.

Until recently, the bootstrap MSBuild was xcopyable, meaning it used to build desktop-targeting projects on a completely empty Windows machine.

At some point this regressed. It is crucial that we continue to be able to have xcopyable MSBuild and we should move towards making MSBuild more portable, not less.

[KirillOsenkov]

Nothing seems to be logging anything into the SdkLogger.

This is never used for anything:

msbuild/src/Build/BackEnd/Components/SdkResolution/SdkResolverService.cs

Line 259 in cc3db35

SdkLogger buildEngineLogger = new SdkLogger(loggingContext);

[KirillOsenkov]

It would be nice to prioritize this, because every release MSBuild ships, it ships being unable to work on an empty machine without the SDK installed. If it continues to be broken, we will be unable to just walk up to the MSBuild repo, build bootstrap and expect that it just works on any machine.

I normally like building and archiving bootstrapped MSBuild for every release, but the last working one I have is from 17.2 or something. This is also useful for bisecting, to answer questions like "which MSBuild release did this break in?"

[KirillOsenkov]

@marcpopMSFT

[KirillOsenkov]

@baronfel I'd love to see this fixed at some point

[KirillOsenkov]

Latest messages from bootstrap MSBuild 17.8.3:

C:\Users\WDAGUtilityAccount\AppData\Local\Microsoft\MSBuild\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.ImportWorkloads.props (14,3): Could not resolve SDK "Microsoft.NET.SDK.WorkloadAutoImportPropsLocator". Exactly one of the probing messages below indicates why we could not resolve the SDK. Investigate and resolve that message to correctly specify the SDK.
  Unable to locate the .NET SDK. Check that it is installed, your PATH is configured for the correct architecture, and that the version specified in global.json (if any) matches the installed version.
  The NuGetSdkResolver did not resolve this SDK because there was no version specified in the project or global.json.
  MSB4276: The default SDK resolver failed to resolve SDK "Microsoft.NET.SDK.WorkloadAutoImportPropsLocator" because directory "C:\Users\WDAGUtilityAccount\AppData\Local\Microsoft\MSBuild\Sdks\Microsoft.NET.SDK.WorkloadAutoImportPropsLocator\Sdk" did not exist.

C:\Users\WDAGUtilityAccount\AppData\Local\Microsoft\MSBuild\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.ImportWorkloads.targets (16,3): Could not resolve SDK "Microsoft.NET.SDK.WorkloadManifestTargetsLocator". Exactly one of the probing messages below indicates why we could not resolve the SDK. Investigate and resolve that message to correctly specify the SDK.
  Unable to locate the .NET SDK. Check that it is installed, your PATH is configured for the correct architecture, and that the version specified in global.json (if any) matches the installed version.
  The NuGetSdkResolver did not resolve this SDK because there was no version specified in the project or global.json.
  MSB4276: The default SDK resolver failed to resolve SDK "Microsoft.NET.SDK.WorkloadManifestTargetsLocator" because directory "C:\Users\WDAGUtilityAccount\AppData\Local\Microsoft\MSBuild\Sdks\Microsoft.NET.SDK.WorkloadManifestTargetsLocator\Sdk" did not exist.

[rainersigwald]

Related to #6566, maybe

msbuild/eng/BootStrapMSBuild.targets

Lines 258 to 260 in 5cf7858

	<!-- Disable workload resolver until we can figure out whether it can work in the bootstrap
	https://github.com/dotnet/msbuild/issues/6566 -->
	<Touch Files="$(BootstrapDestination)\DisableWorkloadResolver.sentinel" AlwaysCreate="true" />

should apply to netfx MSBuild too.

[KirillOsenkov]

@JanKrivanek if there was one MSBuild issue that I'd love to see fixed the most, it's this one

It's super important for standalone bootstrap MSBuild to be able to build projects, and it's currently not working. It's super important to be able to check out any commit of MSBuild, build bootstrap and be able to use that MSBuild to build real projects.

[KirillOsenkov]

I'll venmo $100 to whoever fixes this.

[ozkanpakdil]

This is a long standing issue and I just wanted to dig what is this about(also good motivator 💰 😄 ) I did not understand the full picture but I just download the source of this repo to a new(empty) windows installation and ran the build command, got the error below

  Microsoft.Build.Utilities -> C:\Users\vboxuser\msbuild-main\artifacts\bin\Microsoft.Build.Utilities\Debug\netstandard2.0\Microsoft.Build.Utilities.Core.dll
C:\Users\vboxuser\msbuild-main\.tools\msbuild\17.8.5\tools\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(3345,5): error MSB3821: Couldn't process file system.design\system.design.txt due to its being in the Internet or Res
tricted zone or having the mark of the web on the file. Remove the mark of the web if you want to process these files. [C:\Users\vboxuser\msbuild-main\src\Tasks\Microsoft.Build.Tasks.csproj::TargetFramework=net8.0]
  StringTools -> C:\Users\vboxuser\msbuild-main\artifacts\bin\StringTools\Debug\net35\Microsoft.NET.StringTools.net35.dll
  StringTools.UnitTests -> C:\Users\vboxuser\msbuild-main\artifacts\bin\StringTools.UnitTests\Debug\net472\Microsoft.NET.StringTools.UnitTests.dll
  Microsoft.Build.UnGAC -> C:\Users\vboxuser\msbuild-main\artifacts\bin\Microsoft.Build.UnGAC\Debug\net45\Microsoft.Build.UnGAC.exe
  StringTools.UnitTests -> C:\Users\vboxuser\msbuild-main\artifacts\bin\StringTools.UnitTests\Debug\net8.0\Microsoft.NET.StringTools.UnitTests.dll
  StringTools.UnitTests.net35 -> C:\Users\vboxuser\msbuild-main\artifacts\bin\StringTools.UnitTests.net35\Debug\net472\Microsoft.NET.StringTools.net35.UnitTests.dll
C:\Users\vboxuser\msbuild-main\.tools\msbuild\17.8.5\tools\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(3345,5): error MSB3091: Task failed because "resgen.exe" was not found, or the correct Microsoft Windows SDK is not i
nstalled. The task is looking for "resgen.exe" in the "bin" subdirectory beneath the location specified in the InstallationFolder value of the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v8.0A\WinSDK-NetFx3
5Tools-x86. You may be able to solve the problem by doing one of the following:  1) Install the Microsoft Windows SDK.  2) Install Visual Studio 2010.  3) Manually set the above registry key to the correct location.  4) Pass the correct
 location into the "ToolPath" parameter of the task. [C:\Users\vboxuser\msbuild-main\src\MSBuildTaskHost\MSBuildTaskHost.csproj]
  StringTools.Benchmark -> C:\Users\vboxuser\msbuild-main\artifacts\bin\StringTools.Benchmark\Debug\net472\StringTools.Benchmark.exe
  StringTools.Benchmark -> C:\Users\vboxuser\msbuild-main\artifacts\bin\StringTools.Benchmark\Debug\net8.0\StringTools.Benchmark.dll

Build FAILED.

C:\Users\vboxuser\msbuild-main\src\Directory.Build.targets(137,5): error : TlbExp was not found. Ensure that you have installed everything from .vsconfig. If you have, please report a bug to MSBuild. [C:\Users\vboxuser\msbuild-main\src\
Framework\Microsoft.Build.Framework.csproj::TargetFramework=net472]
C:\Users\vboxuser\msbuild-main\.tools\msbuild\17.8.5\tools\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(3345,5): error MSB3821: Couldn't process file system.design\system.design.txt due to its being in the Internet or Res
tricted zone or having the mark of the web on the file. Remove the mark of the web if you want to process these files. [C:\Users\vboxuser\msbuild-main\src\Tasks\Microsoft.Build.Tasks.csproj::TargetFramework=net8.0]
C:\Users\vboxuser\msbuild-main\.tools\msbuild\17.8.5\tools\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(3345,5): error MSB3091: Task failed because "resgen.exe" was not found, or the correct Microsoft Windows SDK is not i
nstalled. The task is looking for "resgen.exe" in the "bin" subdirectory beneath the location specified in the InstallationFolder value of the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v8.0A\WinSDK-NetFx3
5Tools-x86. You may be able to solve the problem by doing one of the following:  1) Install the Microsoft Windows SDK.  2) Install Visual Studio 2010.  3) Manually set the above registry key to the correct location.  4) Pass the correct
 location into the "ToolPath" parameter of the task. [C:\Users\vboxuser\msbuild-main\src\MSBuildTaskHost\MSBuildTaskHost.csproj]
    0 Warning(s)
    3 Error(s)

Time Elapsed 00:00:25.76
Build failed with exit code 1. Check errors above.

Sorry for my ignorance. Are we trying to fix the this build.cmd process or are we trying to fix the dotnet project generator ?

[miloush]

@ozkanpakdil neither, you should be able to build this repo either way. Try cloning the repo instead of downloading a zip file (or Unblock the zip file in the file properties before extracting the files). The scenario to be fixed is described in the OP, it's running the msbuild itself to compile a blank project.

I run into this issue myself trying to compile a .NET project on a machine that has .NET SDK binaries only, i.e. not installed by the installer.