Remove `Newtonsoft.Json` from the .NET SDK installation

[MichaelSimons]

Summary

The .NET SDK currently ships the Newtonsoft.Json assembly in several locations within the SDK layout. We should remove this dependency from the SDK by eliminating project references that bring it in and migrating usages to System.Text.Json where possible.

Removing this dependency would reduce the SDK servicing surface area, decrease SDK size, and unblock work related to Native AOT.

Motivation

Security / Servicing

Bundling Newtonsoft.Json in the SDK introduces an additional dependency that must be monitored and serviced for potential security vulnerabilities.

Migrating to System.Text.Json reduces this servicing burden because it ships as part of the .NET platform and is already covered by the platform servicing model.

SDK Size Reduction

Removing Newtonsoft.Json would also help reduce the size of the SDK installation.

Historically the SDK contained multiple copies of this assembly. Deduplication work reduced this to two primary copies, but it still contributes unnecessary size to the SDK.

Related work:

• Eliminate Duplicate SDK Files #52182

Native AOT Compatibility

Usage of Newtonsoft.Json is a blocker for some Native AOT scenarios due to its heavy reliance on reflection and dynamic behavior. Migrating to System.Text.Json helps unblock this work.

Related work:

• Enable the dotnet CLI to be shipped as a Native AOT application #49667

Current Locations in the SDK

The following instances of Newtonsoft.Json.dll currently appear in the SDK layout:

• ./DotnetTools/dotnet-user-secrets/11.0.0-preview.3.26152.106/tools/net11.0/any/Newtonsoft.Json.dll - Replace Newtonsoft.Json with System.Text.Json in dotnet-user-secrets aspnetcore#65686
• ./TestHostNetFramework/Newtonsoft.Json.dll
• ./Newtonsoft.Json.dll
• ./Sdks/Microsoft.NET.Sdk/tools/net472/Newtonsoft.Json.dll
• ./Sdks/Microsoft.NET.Sdk.Razor/tools/Newtonsoft.Json.dll - Convert Razor tools from NewtonSoft.Json to System.Text.Json #53345

Migration Guideline

https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/migrate-from-newtonsoft?pivots=dotnet-10-0

[MichaelSimons]

@DustinCampbell, @dotnet/razor-tooling - Regarding the /Microsoft.NET.Sdk.Razor reference, I noticed this code was refactored relatively recently in dotnet/dotnet@d24637c with the comment:

Bring the latest Newtonsoft.Json serialization types from the main Razor repo. This will make future updates to the JSON simpler, since the changes can largely be copied from the Razor repo.

Does Razor have an interest in migrating their code to System.Text.Json to keep this code in sync? Ultimately, I am wondering if someone from Razor can help with migrating the Razor SDK from Newtonsoft.Json to System.Text.Json?

[davidwengier]

@dbreshears FYI ^^^

[MichaelSimons]

@nohwnd - Regarding the ./TestHostNetFramework/Newtonsoft.Json.dll reference, can you comment on the feasibility of migrating the Microsoft.TestPlatform.CLI from Newtonsoft.Json to System.Text.Json? I found microsoft/vstest#2488 which states it won't be implemented. If that is still the case, is the any path forward here?

[nohwnd]

I would like to move it, but it is a potential breaking change, as customers probably depend on us shipping Newtonsoft.Json for them. I did investigate what it would take to move, but would need to spend more time on it to see if it is feasible technically (e.g. we annotate types with data attributes - and system.text.json does not support that, and I am not sure if that detail does not propagate to code that we don't control - hopefully not).

Will discuss it with my manager.

Are there any additional strong reason to drop newtonsoft.json? For me the strongest argument is the first one, security and servicing.

[baronfel]

@nohwnd does the newtonsoft exposure also apply to MTP-based test users? We're looking at AOT'ing the SLK/CLI overall, and part of that is trying to reduce/eliminate usage of dependencies that are not AOT-friendly.

[DustinCampbell]

@DustinCampbell, @dotnet/razor-tooling - Regarding the /Microsoft.NET.Sdk.Razor reference, I noticed this code was refactored relatively recently in dotnet/dotnet@d24637c with the comment:

Bring the latest Newtonsoft.Json serialization types from the main Razor repo. This will make future updates to the JSON simpler, since the changes can largely be copied from the Razor repo.

Does Razor have an interest in migrating their code to System.Text.Json to keep this code in sync? Ultimately, I am wondering if someone from Razor can help with migrating the Razor SDK from Newtonsoft.Json to System.Text.Json?

I'm not working on Razor at the moment, so I can't speak to the team's capacity. However, getting rid of the Newtonsoft.Json dependency would definitely be fantastic.

[nohwnd]

@nohwnd does the newtonsoft exposure also apply to MTP-based test users? We're looking at AOT'ing the SLK/CLI overall, and part of that is trying to reduce/eliminate usage of dependencies that are not AOT-friendly.

it does not, mtp uses system.text.json for .net , and embeds code of Jsonite for .NET Framework.

If there is drive towards NativeAOT then it would be good reason to drive change also into vstest.

[davidwengier]

Razor bits are now using System.Text.Json