Skip to content

[Fusion] Parse batched source results with purpose-built JSON tokenizer - #10644

Merged
tobias-tengler merged 4 commits into
mainfrom
tte/fusion-batched-source-tokenizer
Oct 20, 2026
Merged

tobias-tengler merged 4 commits into
mainfrom
tte/fusion-batched-source-tokenizer

Conversation

@tobias-tengler

@tobias-tengler tobias-tengler commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Batched array responses (variable batching) are the last source documents still parsed with Utf8JsonReader. Each element is parsed through a reader over a ReadOnlySequence of the filled chunks, which is slower than the tokenizer and allocates sequence segments when the chunks are not adjacent.

This PR reads the outer array in the source JSON tokenizer and parses each element into its own document over the shared chunks, through a second generic mode, so single document parses compile to the same code as before. The batched path accepts and rejects the same input as before and creates the same rows, which the tests added in the first commit pin down for every chunk boundary position of each chunk layout. The reader-based parser is removed.

Benchmarks against 7e8896336877

Method Version Mean Ratio Allocated Alloc Ratio
Batch 1 × Small, adjacent base 434.6 ns 1.00 360 B 1.00
Batch 1 × Small, adjacent PR 356.5 ns 0.82 352 B 0.98
Batch 1 × Small, split base 444.4 ns 1.00 360 B 1.00
Batch 1 × Small, split PR 341.2 ns 0.77 352 B 0.98
Batch 1 × Entity, adjacent base 2.66 μs 1.00 360 B 1.00
Batch 1 × Entity, adjacent PR 1.70 μs 0.64 352 B 0.98
Batch 1 × Entity, split base 2.74 μs 1.00 504 B 1.00
Batch 1 × Entity, split PR 1.62 μs 0.59 352 B 0.70
Batch 10 × Small, adjacent base 3.37 μs 1.00 1,392 B 1.00
Batch 10 × Small, adjacent PR 2.51 μs 0.75 1,312 B 0.94
Batch 10 × Small, split base 3.57 μs 1.00 1,488 B 1.00
Batch 10 × Small, split PR 2.42 μs 0.68 1,312 B 0.88
Batch 10 × Entity, adjacent base 25.30 μs 1.00 1,392 B 1.00
Batch 10 × Entity, adjacent PR 16.73 μs 0.66 1,312 B 0.94
Batch 10 × Entity, split base 26.44 μs 1.00 1,680 B 1.00
Batch 10 × Entity, split PR 15.62 μs 0.59 1,312 B 0.78
Batch 100 × Small, adjacent base 33.19 μs 1.00 13,528 B 1.00
Batch 100 × Small, adjacent PR 23.25 μs 0.70 12,728 B 0.94
Batch 100 × Small, split base 34.48 μs 1.00 13,720 B 1.00
Batch 100 × Small, split PR 22.01 μs 0.64 12,728 B 0.93
Batch 100 × Entity, adjacent base 264.78 μs 1.00 14,056 B 1.00
Batch 100 × Entity, adjacent PR 150.10 μs 0.57 12,728 B 0.91
Batch 100 × Entity, split base 264.24 μs 1.00 14,056 B 1.00
Batch 100 × Entity, split PR 147.95 μs 0.56 12,728 B 0.91
Single Apollo, adjacent base 28.91 μs 1.00 272 B 1.00
Single Apollo, adjacent PR 16.70 μs 0.58 264 B 0.97
Single Apollo, split base 29.75 μs 1.00 608 B 1.00
Single Apollo, split PR 15.89 μs 0.53 264 B 0.43
Single Composite, adjacent base 30.93 μs 1.00 1,152 B 1.00
Single Composite, adjacent PR 17.44 μs 0.56 1,064 B 0.92
Single Composite, split base 60.70 μs 1.00 2,784 B 1.00
Single Composite, split PR 17.82 μs 0.29 1,064 B 0.38
Single Escaped, adjacent base 72.57 μs 1.00 272 B 1.00
Single Escaped, adjacent PR 45.37 μs 0.63 264 B 0.97
Single Escaped, split base 76.17 μs 1.00 560 B 1.00
Single Escaped, split PR 46.85 μs 0.62 264 B 0.47
Single Large, adjacent base 97.89 μs 1.00 656 B 1.00
Single Large, adjacent PR 51.33 μs 0.52 264 B 0.40
Single Large, split base 97.48 μs 1.00 656 B 1.00
Single Large, split PR 53.00 μs 0.54 264 B 0.40
Single LongStrings, adjacent base 2.89 μs 1.00 272 B 1.00
Single LongStrings, adjacent PR 2.37 μs 0.82 264 B 0.97
Single LongStrings, split base 3.02 μs 1.00 560 B 1.00
Single LongStrings, split PR 2.62 μs 0.87 264 B 0.47
Single Numbers, adjacent base 170.22 μs 1.00 272 B 1.00
Single Numbers, adjacent PR 92.78 μs 0.55 264 B 0.97
Single Numbers, split base 186.67 μs 1.00 608 B 1.00
Single Numbers, split PR 92.80 μs 0.50 264 B 0.43
Single Pretty, adjacent base 50.15 μs 1.00 272 B 1.00
Single Pretty, adjacent PR 27.59 μs 0.55 264 B 0.97
Single Pretty, split base 50.01 μs 1.00 608 B 1.00
Single Pretty, split PR 28.38 μs 0.57 264 B 0.43
Single Small, adjacent base 365.3 ns 1.00 272 B 1.00
Single Small, adjacent PR 275.7 ns 0.75 264 B 0.97
Single Small, split base 365.3 ns 1.00 272 B 1.00
Single Small, split PR 271.1 ns 0.74 264 B 0.97

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Patch coverage

94.3% of changed lines covered (132/140)

File Covered Changed Patch %
…/Fusion.Execution/Text/Json/SourceResultDocument.Tokenizer.cs 101 109 92.7% 🟡
…/AspNetCore/src/Transport.Http/Json/JsonResultEnumerable.cs 1 1 100.0% 🟢
…/src/Fusion.Execution/Text/Json/SourceResultDocument.Parse.cs 30 30 100.0% 🟢
Uncovered changed lines (JSON)
{
  "sha": "bd12a1f944d89905597162c67e84ee5938c11321",
  "files": [
    { "path": "src/HotChocolate/Fusion/src/Fusion.Execution/Text/Json/SourceResultDocument.Tokenizer.cs", "ranges": [[514, 515], [523, 524], [1154, 1155], [1158, 1159]] }
  ]
}

Project coverage: 59.1% (312918/529175 lines)

@tobias-tengler
tobias-tengler force-pushed the tte/fusion-batched-source-tokenizer branch from af1b632 to b84bcd4 Compare October 20, 2026 13:13
@tobias-tengler
tobias-tengler marked this pull request as ready for review October 20, 2026 14:18
Copilot AI balanced review requested due to automatic review settings October 20, 2026 14:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@tobias-tengler
tobias-tengler force-pushed the tte/fusion-batched-source-tokenizer branch from c1a376d to 8349639 Compare October 20, 2026 14:50
Base automatically changed from tte/fusion-source-json-tokenizer to main October 20, 2026 15:22
@tobias-tengler tobias-tengler changed the title [Fusion] Parse batched source results with the source JSON tokenizer [Fusion] Parse batched source results with purpose-built JSON tokenizer Oct 20, 2026
The batched array path gets a test per behavior: the rows of each
element, empty arrays, input after the array, the depth limit, and the
arrays it rejects, each parsed at every chunk boundary position in each
chunk layout. The JSON result enumerable is tested end to end.
Batched array responses were the last source documents parsed with
Utf8JsonReader over a ReadOnlySequence of the filled chunks. The tokenizer
now reads the outer array and parses each element into its own document
over the shared chunks, which removes the reader-based parser. Single
document parses compile to the same code as before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants