[main] Update dotnet/runtime by lewing · Pull Request #35048 · dotnet/sdk

lewing · 2023-08-30T17:52:22Z

Fix the 9.0.100 band in the emsdk transport package name

lewing · 2023-08-30T17:54:34Z

this may need additional tweaks to deal with the band versioning

lewing · 2023-08-30T18:17:20Z

Installer needs a bunch of updates as well once this lands

jeffschwMSFT · 2023-08-30T20:43:05Z

who is on point to either identifying the individual failures and farming out the work and/or handling them?

mmitche · 2023-08-30T20:57:34Z

@jeffschwMSFT There shouldn't be too much. This was the commit to switch to 8: 8119940

lewing · 2023-08-30T21:34:13Z

opened dotnet/arcade#14002 for future me

lewing · 2023-08-30T22:22:55Z

Looks like it is failing on rid resolution now @elinor-fung can you take a look?

elinor-fung · 2023-08-30T23:43:59Z

The built tests are configured to run on a runtime from before dotnet/docs#36466 for some reason.

If I build main, the runtimeconfig for the Microsoft.NET.Build.Tests has the runtime version corresponding to Version.props/Version.Details.xml:

  "runtimeOptions": {
    "tfm": "net8.0",
    "rollForward": "Major",
    "framework": {
      "name": "Microsoft.NETCore.App",
      "version": "8.0.0-rc.1.23421.3"
    },

But from this branch, it has:

  "runtimeOptions": {
    "tfm": "net8.0",
    "rollForward": "Major",
    "framework": {
      "name": "Microsoft.NETCore.App",
      "version": "8.0.0-preview.7.23375.6"
    },

I'm not sure why yet - cc @dsplaisted in case it is something obvious to you

elinor-fung · 2023-08-31T00:03:31Z

This is what does the updating to the runtime version that gets used by tests:

sdk/Directory.Build.targets

Lines 46 to 54 in aabbd97

    
             <!-- Update KnownFrameworkReferences to target the right version of the runtime --> 
        
             <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETCoreApp' 
        
                                     and $(MicrosoftNETCoreAppRefPackageVersion.StartsWith('$(_TargetFrameworkVersionWithoutV)')) 
        
                                     and '$(MSBuildProjectName)' != 'toolset-tasks'"> 
        
               <FrameworkReference 
        
                   Update="Microsoft.NETCore.App" 
        
                   TargetingPackVersion="$(MicrosoftNETCoreAppRefPackageVersion)" 
        
                   RuntimeFrameworkVersion="$(MicrosoftNETCoreAppRuntimePackageVersion)" /> 
        
             </ItemGroup>

But with this PR, $(MicrosoftNETCoreAppRefPackageVersion.StartsWith('$(_TargetFrameworkVersionWithoutV)')) is false:

MicrosoftNETCoreAppRefPackageVersion is 9.0.0-alpha.1.23430.1
_TargetFrameworkVersionWithoutV is 8.0

elinor-fung · 2023-08-31T00:12:09Z

@dsplaisted / @marcpopMSFT how does SDK usually handle updating major versions and targeting the consumed runtime version?

The retargeting for ASP.NET references has a similar condition:

sdk/Directory.Build.targets

Lines 56 to 63 in aabbd97

    
           <ItemGroup Condition="$(MicrosoftAspNetCoreAppRefPackageVersion.StartsWith('$(_TargetFrameworkVersionWithoutV)'))"> 
        
             <KnownFrameworkReference Update="Microsoft.AspNetCore.App"> 
        
               <LatestRuntimeFrameworkVersion>$(MicrosoftAspNetCoreAppRefPackageVersion)</LatestRuntimeFrameworkVersion> 
        
               <RuntimePackRuntimeIdentifiers>${SupportedRuntimeIdentifiers}</RuntimePackRuntimeIdentifiers> 
        
               <TargetingPackVersion>$(MicrosoftAspNetCoreAppRefPackageVersion)</TargetingPackVersion> 
        
               <DefaultRuntimeFrameworkVersion>$(MicrosoftAspNetCoreAppRefPackageVersion)</DefaultRuntimeFrameworkVersion> 
        
             </KnownFrameworkReference> 
        
           </ItemGroup>

dsplaisted · 2023-08-31T16:30:02Z

Now that the dependency flow is bringing in .NET 9 instead of .NET 8, the build doesn't have a version of .NET 8 to update the targeting to. I think the thing to do here is to update the stage 0 SDK to one that has a recent enough version of the runtime.

FYI for those not aware, here is where we are trying to keep track of the changes needed each time we update to a new TargetFramework: https://github.com/dotnet/sdk/blob/main/documentation/general/UpdateToNewTargetFramework.md Feel free to update it as appropriate.

lewing · 2023-09-02T02:06:28Z

@ViktorHofer @dotnet/domestic-cat please take a look

baronfel · 2023-09-13T18:40:57Z

+            var originalBundledNETCoreAppPackageVersion = propertyGroup.Element(ns + "BundledNETCoreAppPackageVersion").Value;
+            var parsedOriginalBundledPackageVersion = SemanticVersion.Parse(originalBundledNETCoreAppPackageVersion);
+            var parsedMicrosoftNETCoreAppRefPackageVersion =
+                SemanticVersion.Parse(microsoftNETCoreAppRefPackageVersion);
+
+            // In the case where we have a new major version, it'll come in first through the dotnet/runtime flow of the
+            // SDK's own package references. The Stage0 SDK's bundled version props file will still be on the older major version
+            // (and the older TFM that goes along with that). If we just replaced the bundled version with the new major version,
+            // apps that target the 'older' TFM would fail to build. So we need to keep the bundled version from the existing
+            // bundledversions.props in this one specific case.
+
+            var newBundledPackageVersion =
+                parsedOriginalBundledPackageVersion.Major == parsedMicrosoftNETCoreAppRefPackageVersion.Major
+                ? microsoftNETCoreAppRefPackageVersion
+                : originalBundledNETCoreAppPackageVersion;
+
+            propertyGroup.Element(ns + "BundledNETCoreAppPackageVersion").Value = newBundledPackageVersion;



@dsplaisted would you mind double-checking this and seeing if it matched what you were talking about?

Yes, this matches what I expected. It might be better to use NuGetVersion instead of SemanticVersion. I'm not too sure of the differences between them, but the code I've seen has used NuGetVersion.

marcpopMSFT · 2023-09-14T17:18:29Z

latest change has us down to 34 test failures. @dotnet/source-build-internal @agocke @sbomer

Microsoft.NET.Sdk.BlazorWebAssembly.AoT.Tests.WasmAoTPublishIntegrationTest.AoT_Publish_WithExistingWebConfig_Works
emcc : error : C:\h\w\B1C3097B\t\dotnetSdkTests\ij4x4eyn.d0b\.nuget\packages\microsoft.netcore.app.runtime.mono.browser-wasm\8.0.0-rc.1.23414.4\runtimes\browser-wasm\native\libmono-wasm-simd.a: No such file or directory ("C:\h\w\B1C3097B\t\dotnetSdkTests\ij4x4eyn.d0b\.nuget\packages\microsoft.netcore.app.runtime.mono.browser-wasm\8.0.0-rc.1.23414.4\runtimes\browser-wasm\native\libmono-wasm-simd.a" was expected to be an input file, based on the commandline arguments provided) [C:\h\w\B1C3097B\t\dotnetSdkTests\ij4x4eyn.d0b\AoT_Publish_W---DA56BF44\blazorwasm\blazorwasm.csproj]
Microsoft.NET.Publish.Tests.GivenThatWeWantToPublishAnAotApp.NativeAotLib_errors_out_when_eventpipe_is_enabled
Expected command to fail but it did not.
Source build is flagging the 8.0 downgrades I checked in to fix the System.Text.Json failure. Not sure what to do about this as it's meant to be temporary

Package IDs are:
Microsoft.Bcl.AsyncInterfaces.8.0.0-rc.1.23414.4
Microsoft.Extensions.DependencyModel.8.0.0-rc.1.23414.4
Microsoft.NET.HostModel.8.0.0-rc.1.23414.4```

There are a few other individual tests failures are unbuntu it looks like that also need to be looked into.

elinor-fung · 2023-09-14T19:06:41Z

Microsoft.NET.Publish.Tests.GivenThatWeWantToPublishAnAotApp.NativeAotLib_errors_out_when_eventpipe_is_enabled

Microsoft.DotNet.ILCompiler being used is now 8.0.0-rc.1.23414.4 - which is from before dotnet/runtime#90811 (main is using 8.0.0-rc.1.23421.3). Maybe related to the BundledVersions changes in this PR?

Side note: not the issue being hit in this PR, but that error is also being downgraded to a warning (dotnet/runtime#91715), so the test will also need to be updated when that change flows over.

lewing · 2023-09-15T16:00:18Z

I've updated the runtime flow to the most recent build. The wasm build is odd @maraf and @radical can you please take a look.

elinor-fung · 2023-09-15T17:03:49Z

C:\h\w\B1C3097B\t\dotnetSdkTests\ij4x4eyn.d0b.nuget\packages\microsoft.netcore.app.runtime.mono.browser-wasm\8.0.0-rc.1.23414.4\runtimes\browser-wasm\native\libmono-wasm-simd.a: No such file or directory

Maybe the wasm error is the same as root cause as the native aot one - seems like the test environment gets constructed such that it is using older packages. As with Microsoft.DotNet.ILCompiler, the version of Microsoft.NETCore.App.Runtime.Mono.browser-wasm being used is an older version - 8.0.0-rc.1.23414.4 - which is before libmono-wasm-simd.a was added.

radical · 2023-09-15T18:35:31Z

C:\h\w\B1C3097B\t\dotnetSdkTests\ij4x4eyn.d0b.nuget\packages\microsoft.netcore.app.runtime.mono.browser-wasm\8.0.0-rc.1.23414.4\runtimes\browser-wasm\native\libmono-wasm-simd.a: No such file or directory

This is because 9.0 targets are being used with a 8.0 runtime pack. I'm trying to reproduce this locally now.

The tests fail with: ``` emcc : error : /private/tmp/helix/working/A19408C3/w/B54A09BA/e/testExecutionDirectory/.nuget/packages/microsoft.netcore.app.runtime.mono.browser-wasm/8.0.0-rc.1.23414.4/runtimes/browser-wasm/native/libmono-wasm-simd.a: No such file or directory ``` .. because the build is using 9.0 targets, but 8.0 runtime pack. This commit works around that issue by overriding the runtime pack path to point to the correct one. `WasmOverridePacks.targets` is taken from `Wasm.Build.Tests` in `dotnet/runtime`. This workaround can be removed once we have a proper 9.0 workload.

lewing · 2023-09-18T17:16:29Z

Looks like we are down to NativeAOT publish failures now?

marek-safar · 2023-09-18T18:20:16Z

@sbomer @LakshanF could you help with resolving tests failures in AotStaticLibraryPublishWithEventPipe ?

marek-safar · 2023-09-19T09:48:57Z

@dotnet/source-build-internal could you please advise how to track where these prebuilts are coming from and how to version baseline-comparison.xml in 8.0 arcade for 9.0 main?

MichaelSimons · 2023-09-19T13:52:36Z

    </Dependency>
-    <Dependency Name="Microsoft.NETCore.DotNetHostResolver" Version="8.0.0-rc.1.23421.3">
+    <Dependency Name="Microsoft.NETCore.DotNetHostResolver" Version="9.0.0-alpha.1.23455.1">
      <Uri>https://github.com/dotnet/runtime</Uri>


Is this intentionally incoherent with the other runtime dependencies? They are on version 9.0.0-alpha.1.23464.19 and sha a6c64c3c26417f579022e20cbcee992b4814ed2c. This is the source of the following source-build prebuilts that are being reported:

<Usage Id="Microsoft.NETCore.DotNetAppHost" Version="9.0.0-alpha.1.23455.1" File="src/artifacts/obj/Microsoft.DotNet.MSBuildSdkResolver/project.assets.json" /> <Usage Id="Microsoft.NETCore.DotNetAppHost" Version="9.0.0-alpha.1.23455.1" File="src/artifacts/obj/Microsoft.DotNet.SdkResolver/project.assets.json" /> <Usage Id="Microsoft.NETCore.DotNetHostResolver" Version="9.0.0-alpha.1.23455.1" File="src/artifacts/obj/Microsoft.DotNet.MSBuildSdkResolver/project.assets.json" IsDirectDependency="true" /> <Usage Id="Microsoft.NETCore.DotNetHostResolver" Version="9.0.0-alpha.1.23455.1" File="src/artifacts/obj/Microsoft.DotNet.SdkResolver/project.assets.json" IsDirectDependency="true" />

We actually removed Microsoft.NETCore.DotNetHostResolver in runtime. It looks like this repo is using it to grab hostfxr for Framework builds. I think it can use Microsoft.NETCore.App.Runtime.* packages - I'll take a look.

MichaelSimons · 2023-09-19T14:19:36Z

@dotnet/source-build-internal could you please advise how to track where these prebuilts are coming from and how to version baseline-comparison.xml in 8.0 arcade for 9.0 main?

I took a quick look. Here is what I did:

Reviewed the prebuilt baseline-comparison.xml.
Noted two classes of prebuilts as they come from two different project groupings.
First grouping is from src/artifacts/obj/Release/Sdks/Microsoft.NET.Sdk/tools/project.assets.json which when looking at the changes led me to this conclusion.
Second grouping is from src/artifacts/obj/Microsoft.DotNet.MSBuildSdkResolver/project.assets.json and src/artifacts/obj/Microsoft.DotNet.SdkResolver/project.assets.json which led me to review the version of the referenced prebuilts in the versions.props and version.details.xml and noticed this incoherency.

elinor-fung · 2023-09-19T19:23:45Z

Second grouping from Microsoft.DotNet.MSBuildSdkResolver / Microsoft.DotNet.SdkResolver is resolved.

#35048 (comment) is all that is left.

lewing · 2023-09-19T20:38:24Z

@marcpopMSFT ping

[main] Update dotnet/runtime

5a7e1f2

lewing requested review from dsplaisted, marcpopMSFT and mmitche August 30, 2023 17:53

lewing requested a review from agocke August 30, 2023 17:54

mmitche approved these changes Aug 30, 2023

View reviewed changes

mmitche reviewed Aug 30, 2023

View reviewed changes

Comment thread eng/Versions.props Outdated

Fix SharedFramework too

ac96fe8

lewing added the Area-CodeFlow label Aug 30, 2023

hoyosjs approved these changes Aug 30, 2023

View reviewed changes

ghost added the untriaged Request triage from a team member label Aug 30, 2023

joeloff approved these changes Aug 30, 2023

View reviewed changes

Update the variable used in global.json too

09e6547

Jason Zhai added 2 commits August 31, 2023 01:34

Update global.json

8bed3cf

Update blazor web tag

98927b2

v-wuzhai requested a review from a team as a code owner August 31, 2023 10:24

Jason Zhai and others added 3 commits August 31, 2023 20:14

Update blazor web tag for osx and linux

ab87475

Update global.json

ef8f565

Bump runtime dependencies

f8f8fc2

baronfel reviewed Sep 13, 2023

View reviewed changes

marcpopMSFT added 2 commits September 13, 2023 15:09

Add an additional dependency that's flagged in the tests.

f8ef959

Remove the cryptography reference per PR feedback

a1ee049

Bump runtime build

4ecccd1

radical added 2 commits September 15, 2023 16:09

Use WasmOverride.targets only for net8.0 wasm projects

ef678de

Merge branch 'main' into update-emsdk-manifest

ab71881

LakshanF mentioned this pull request Sep 18, 2023

Add a EventPipe test for a native AOT library #34839

Closed

Fix native aot lib test scenario

ce306be

Update versions to official RC1

d1f1784

MichaelSimons reviewed Sep 19, 2023

View reviewed changes

Comment thread src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj

Remove Microsoft.NETCore.DotNetHostResolver dependency

ca8ddf9

lewing mentioned this pull request Sep 19, 2023

[main] Update dependencies from dotnet/runtime #35493

Closed

Update SourceBuildPrebuiltBaseline.xml for 8.0 msbuild dependencies

26162bd

marcpopMSFT enabled auto-merge September 19, 2023 22:53

marcpopMSFT merged commit 1bdc5a5 into main Sep 20, 2023

marcpopMSFT deleted the update-emsdk-manifest branch September 20, 2023 06:59

ellahathaway mentioned this pull request Nov 28, 2023

Use .NET 9.0 versions for task dependencies #37013

Merged

Uh oh!

Conversation

lewing commented Aug 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lewing commented Aug 30, 2023

Uh oh!

Uh oh!

lewing commented Aug 30, 2023

Uh oh!

jeffschwMSFT commented Aug 30, 2023

Uh oh!

mmitche commented Aug 30, 2023

Uh oh!

lewing commented Aug 30, 2023

Uh oh!

lewing commented Aug 30, 2023

Uh oh!

elinor-fung commented Aug 30, 2023

Uh oh!

elinor-fung commented Aug 31, 2023

Uh oh!

elinor-fung commented Aug 31, 2023

Uh oh!

dsplaisted commented Aug 31, 2023

Uh oh!

lewing commented Sep 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

baronfel Sep 13, 2023

Choose a reason for hiding this comment

Uh oh!

dsplaisted Sep 14, 2023

Choose a reason for hiding this comment

Uh oh!

marcpopMSFT commented Sep 14, 2023

Uh oh!

elinor-fung commented Sep 14, 2023

Uh oh!

lewing commented Sep 15, 2023

Uh oh!

elinor-fung commented Sep 15, 2023

Uh oh!

radical commented Sep 15, 2023

Uh oh!

lewing commented Sep 18, 2023

Uh oh!

marek-safar commented Sep 18, 2023

Uh oh!

marek-safar commented Sep 19, 2023

Uh oh!

MichaelSimons Sep 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elinor-fung Sep 19, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MichaelSimons commented Sep 19, 2023

Uh oh!

elinor-fung commented Sep 19, 2023

Uh oh!

lewing commented Sep 19, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

18 participants

lewing commented Aug 30, 2023 •

edited

Loading

lewing commented Sep 2, 2023 •

edited

Loading

MichaelSimons Sep 19, 2023 •

edited

Loading