perf(server): bounded snapshots of threads with large messages are prepared faster

[bompus]

Preparing a bounded snapshot of a thread with ten 1 MiB messages takes 21.2 ms, was 39.2 ms (46% faster), with 42% less CPU per 10 requests. Peak memory did not improve.

Problem

A bounded snapshot still serializes its complete projection to calculate payloadBudgetExceeded when control state fits the byte budget. Large retained messages can already prove overflow, but the server allocates and encodes the timeline anyway. Local timeline items appear twice in this projection before wire compaction.

Change

Use text lengths as a conservative lower bound on encoded bytes. Count text in visibleTurnItems and turnItems, including both local copies. If the sum exceeds the budget, report overflow immediately. Otherwise retain the exact JSON byte calculation.

Page selection, complete turns, cursors, response fields and validation stay unchanged. No dependencies or migrations.

Scope and approval

Closes #18916.

Focused follow-up to #17387's overflow calculation, independent of #18071's ancestor-query optimization. Related #16819 and #15794 address page-selection limits; this change preserves those limits and optimizes the final overflow check.

Replaces #17426, which was closed because its body did not name the model and harness. The code is unchanged; they are stated at the end.

Verification

A synthetic read-only SQLite comparison used Node 24.21.0 and Effect 4.0.1. The large-message case retains ten 1 MiB assistant messages. Timed work includes SQL, validated decoding, wire projection, bounding, schema encoding and JSON serialization. Startup, hashing, network transport and client rendering are outside request timing. Hashing remains included in batch CPU.

Large-message case	Baseline	With shortcut	Benefit
Median preparation time	39.19 ms	21.17 ms	46% lower
Median CPU per 10-request batch	485.32 ms	281.26 ms	42% lower
Median fresh-process peak RSS	594.5 MiB	597.9 MiB	No clear change
Encoded response size	10,696,937 bytes	10,696,937 bytes	Unchanged

Changes of 3% or less are treated as no clear change in this comparison. This is a practical reporting threshold, not a statistical margin of error. A larger change must still exceed measured variability.

Peak RSS did not improve. Four ordinary controls stayed within the preset 5% latency/CPU regression limit; their differences do not establish a speedup.

Each case used four warmup pairs and 15 alternating measured pairs, with ten requests per arm per pair. The target's paired median saving was 17.92 ms; paired-delta median absolute deviation was 1.13 ms. RSS used three alternating pairs of fresh processes, each with ten warmup and 80 measured requests. All 2,440 responses matched exactly, including flags and cursors; fixtures were unchanged.

The measured baseline was 8ba4bdb1f6, including the independent #18071 optimization, plus this exact patch for the candidate. The final branch was rebased onto 43f8a8de17 with byte-identical affected source/tests. This does not measure performance against current main or establish a production/UI speedup.

Real-client rendering was not measured. The full repository suite was not run locally; upstream CI passed on b6cd4175db.

68 tests passed on the final branch. New coverage checks exact budget boundaries for ASCII, JSON escapes, Unicode, unpaired surrogates, inherited/local rows, and combined duplicated text while preserving complete turns and cursors.

• Server typecheck, targeted lint/format, diff check and test floor guard passed.

Focused test command

vp test run apps/server/src/orchestration-v2/threadHistoryPaging.test.ts apps/server/src/orchestration-v2/WireProjection.test.ts apps/server/src/orchestration-v2/ThreadStream.test.ts

GPT-6.1 Sol, in Codex.

[macroscopeapp]

Approvability

Verdict: Approved at b6cd417

Macroscope's review found this PR approvable — This is a focused server-side performance optimization that skips expensive full JSON encoding only when timeline text already proves the existing payload budget is exceeded. It leaves projection data and pagination behavior unchanged and adds targeted boundary coverage.

Notes:

• All code in this push has already been reviewed. Approvability was decided on eligibility alone.

^{You can add or adjust custom eligibility rules. Learn more.}

[coderabbitai]

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

⚙️ Run configuration

• Configuration used: Path: .coderabbit.config.ts
• Review profile: CHILL
• Plan: Advanced
• Run ID: 0fb88083-bb30-4e06-8846-45c2d9a9acba

📥 Commits

Reviewing files that changed from the base of the PR and between b6cd417 and 1e9e973.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Path: .coderabbit.config.ts
• Review profile: CHILL
• Plan: Advanced
• Run ID: 3ce06edc-e589-4651-a0f6-64f0b0b83410

📥 Commits

Reviewing files that changed from the base of the PR and between 77c8747 and b6cd417.

📒 Files selected for processing (2)

• apps/server/src/orchestration-v2/threadHistoryPaging.test.ts
• apps/server/src/orchestration-v2/threadHistoryPaging.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

📝 Walkthrough

📝 Walkthrough

Walkthrough

The thread history projection now checks timeline text length before serializing the full projection for its payload-budget result. Tests cover multiple text encodings, local and inherited rows, and a large user-assistant turn.

Changes

Thread history payload budget

Layer / File(s)	Summary
Timeline text budget check apps/server/src/orchestration-v2/threadHistoryPaging.ts, apps/server/src/orchestration-v2/threadHistoryPaging.test.ts	The projection sums UTF-16 text lengths across visible rows and turn items, including duplicated local items, and checks for overflow before full serialization. Tests cover text encodings, row origins, budget boundaries, and a large user-assistant turn. They also verify the retained projection and paging metadata.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Refactor

Suggested reviewers: yash-singh1

Merge Risk: ⚪ Minimal · up to b6cd4

No actionable payload-budget risk remains. The change is mergeable after normal checks.

### Architecture Summary

Architecture risk: 🔵 Low · up to b6cd4

The change affects 1 system.

Changed systems: apps/server

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

• observed — apps/server (service) was modified; 2 changed files map to changed impact.

Before / after behavior

• observed — Modified behavior in apps/server/src/orchestration-v2/threadHistoryPaging.test.ts: The test import adds vi for spying on JSON.stringify.
• observed — Modified behavior in apps/server/src/orchestration-v2/threadHistoryPaging.test.ts: Adds parameterized expectations for overflow behavior with four text encodings and local versus inherited rows. For budgets equal to the initial serialized size, one byte below it, or 6,000 bytes, the test checks the overflow flag, retained projection and paging metadata, and whether serialization is attempted based on a minimum text-byte estimate.
• observed — Modified behavior in apps/server/src/orchestration-v2/threadHistoryPaging.test.ts: Adds a test expecting a large user question and assistant answer to remain together in the bounded projection while combined duplicated text sets payloadBudgetExceeded; it also expects no history cursor or additional-history flag.
• observed — Modified behavior in apps/server/src/orchestration-v2/threadHistoryPaging.ts: Added timelineTextExceedsBudget, which checks string text in visible rows and turn items, counts both collections, and exits once their combined UTF-16 length exceeds the budget.

Pre-merge checks: 4

Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Title check		The title clearly identifies the server performance improvement for bounded snapshots with large messages. It accurately summarizes the primary user-visible effect of the change.
Description check		The description includes complete Problem, Change, Scope and approval, and Verification sections. It explains the implementation, linked issues, measured results, test coverage, limitations, and agent…

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

Dismissing prior approval to re-evaluate b6cd417

[maria-rcks]

Note

Written by gpt-6.1-sol on behalf of Maria

Thanks for investigating snapshot serialization costs. Under our limited review capacity in Read this first, right now we are only taking focused bug fixes from outside contributors. This PR preserves snapshot behavior while reducing serialization work, and its synthetic measurements exclude transport and client rendering. Please keep #18916 updated and seek agreement in an Ideas discussion on a measured user-path fix before requesting reconsideration.