Fix Utf8JsonReader invalid-literal errors: unbounded message, dropped bytes, IndexOutOfRangeException

[caraioniurie47]

When Utf8JsonReader finds an invalid true, false or null, the exception depends on how the input is segmented, and in some layouts it is not a JsonException at all. Measured on .NET 10.0.12:

• Single segment (byte[], or a one-segment ReadOnlySequence): the message quotes everything to the end of the buffer. CheckLiteral passes ThrowInvalidLiteral the span ConsumeLiteral gave it, _buffer.Slice(_consumed), so tb: gives 'tb:' is an invalid JSON literal, and a 1,000,007-byte payload starting [falsX gave a 1,000,105-character message.
• Multi-segment: a truncated literal can throw IndexOutOfRangeException, e.g. segments "t", "" or "[", "tr", "". CheckLiteralMultiSegment copied a span into its message buffer only after GetNextSpan() returned the next one, so nothing was copied, and GetInvalidLiteralMultiSegment read span[0] of an empty span.
• Multi-segment: the last segment's bytes are dropped from the message when the data ends inside the literal, for the same reason: fals read one byte per segment gives 'fal'.
• Multi-segment: BytesConsumed after the exception can be too low when the literal straddles a segment boundary: [tru as two segments split after [t or [tr reports 0, while the byte[] and the other splits that throw a JsonException report 1. The throw path restored _totalConsumed but not _consumed, which GetNextSpan() had reset.

Change. Both paths now quote the literal's bytes up to and including the first mismatching byte, or every byte present when the data ends, so the quote is at most the literal's length and the same for every segmentation. CheckLiteralMultiSegment copies each span's matching bytes as soon as they match, which removes the AmountToWrite helper, and restores _consumed before throwing. How BytePositionInLine and LineNumber are computed is unchanged.

This addresses the literal half of #30706. The comment-handling differences listed there ({/, /+ and others, with JsonCommentHandling.Skip and Allow) are not changed here and remain. Nothing is thrown that was not thrown before, apart from JsonException replacing IndexOutOfRangeException; the observable differences are shorter messages, and a message, BytesConsumed and Position that are the same across segmentations.

Tests

• InvalidLiteralVariousSegmentSizes added to Utf8JsonReaderTests.MultiSegment.cs, in the shape of InvalidJsonNumberVariousSegmentSizes: 11 inputs, each read as a byte[], a one-segment ReadOnlySequence, one byte per segment (the helper adds an empty last segment), and every one- and two-split ReadOnlySequence, including splits at the end so an empty last segment is covered. Each read asserts LineNumber, BytePositionInLine, BytesConsumed, Position and the quoted literal.
• Without the change all 11 fail, in both test target frameworks: IndexOutOfRangeException for t, 'nu', 'fal' and 'tr' for the dropped bytes, 'tb:' and 'nulL]' for the whole-buffer quote. With it all 11 pass.
• With the change, System.Text.Json.Tests, System.Text.Json.SourceGeneration.Roslyn4.4.Tests and System.Text.Json.SourceGeneration.Roslyn3.11.Tests pass in both target frameworks.

Performance

BenchmarkDotNet 0.15.2, Windows 11 x64: a Release build of main and the same build with this change to System.Text.Json, each in its own Release testhost, compared with --coreRun. The benchmark reads an array of 3000 elements to the end: Literals is true,false,null repeated, Objects is {"id":n,"active":true|false,"parent":null,"name":"item n"}. Segment size 0 is a byte[]; with 3 every literal straddles segments, the path whose copying changed. Three runs, one launch per build and then three, mean ± SD in µs:

Payload	Segment	main (runs 1 / 2 / 3)	This branch (runs 1 / 2 / 3)
Literals	0	34.08 ± 0.13 / 33.21 ± 0.72 / 32.35 ± 0.60	32.74 ± 0.08 / 32.29 ± 0.32 / 33.13 ± 0.90
Literals	3	176.34 ± 0.74 / 170.64 ± 6.28 / 170.69 ± 4.76	172.55 ± 0.86 / 175.85 ± 2.82 / 172.25 ± 1.28
Literals	64	46.50 ± 0.11 / 46.90 ± 4.64 / 46.99 ± 0.91	40.93 ± 0.15 / 48.66 ± 0.85 / 48.61 ± 1.34
Literals	4096	38.40 ± 0.07 / 37.60 ± 0.66 / 39.79 ± 3.77	38.75 ± 0.06 / 38.68 ± 0.27 / 38.67 ± 0.98
Objects	0	348.38 ± 2.88 / 341.95 ± 6.41 / 335.60 ± 2.24	339.04 ± 0.82 / 343.34 ± 5.74 / 334.26 ± 4.94
Objects	3	1469.70 ± 10.66 / 1458.81 ± 28.13 / 1431.73 ± 17.36	1426.20 ± 5.55 / 1445.56 ± 16.71 / 1422.44 ± 6.70
Objects	64	480.26 ± 1.30 / 475.91 ± 15.57 / 473.75 ± 2.57	504.10 ± 1.28 / 503.63 ± 6.53 / 472.93 ± 18.67
Objects	4096	397.41 ± 4.68 / 390.79 ± 9.30 / 392.40 ± 9.09	388.64 ± 1.76 / 390.11 ± 5.88 / 396.80 ± 8.71

By mean, the lower build changes between runs in every row except Objects/3, where this branch is lower in all three runs by 0.6% to 3%. Objects/64, higher on this branch in runs 1 and 2, was rerun alone with twenty launches per build, once in each host order: this branch 485.7 ± 18.3 against main 481.0 ± 19.4 µs, and 490.9 ± 16.5 against 491.0 ± 15.3 µs. Its spread is about 3.5% of the mean and neither build is consistently ahead. The byte[] rows, whose only change is in the throw path, move by up to 4% between builds within a run.

Contributes to #30706

Note

AI-generated, written at my direction and reviewed by me before posting. Everything ran on Windows 11 x64 (build 26200). The measurements in the first list ran on the installed .NET 10.0.12 release with a scratch console app; the two crash inputs and the 1,000,105-character message were re-run under the Release corerun of main built locally, with the same result, and under this branch's, where they give JsonException and a 104-character message. Position after the exception was measured the same way, on all three builds, for every two-segment split of [tru, [nulL], {"":tr and [ trux]. The tests ran on Debug libraries with a Checked runtime; the benchmark as described under Performance.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json
See info in area-owners.md if you want to be subscribed.