[macandro96]

What does this PR do ?

Batch generation-prefix writes to TransferQueue during prefix-level rollout checkpoint cuts.

This is a stacked PR based on amahishi/prefix-recovery-core. It:

• adds configurable per-worker row and token limits for generation-prefix batches;
• moves the batching transaction into the shared generation-cut capture mixin introduced by the base branch;
• adds a batched TQTokenSink API that preserves jagged token fields and separates incompatible optional schemas;
• validates every result before sealing worker capture buffers and rolls back or clears rejected rows on failure;
• retains the single-row path when the configured batch size is 1;
• adds unit coverage for ragged rows, optional routing payloads, batch failures, lifecycle races, repeated cuts, and large inventories;
• adds focused proxy and TQ benchmark tools for measuring checkpoint write scaling.

Issues

This is part of the prefix-level rollout checkpointing stack.

Usage

Configure the checkpoint batching limits under rollout_checkpointing:

generation_prefix_batch_size: 256
generation_prefix_batch_max_tokens: 4194304

The generation owner gathers active prefixes until either limit is reached, then performs one TQ put_samples operation for each compatible schema group.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

Local validation completed:

• Ruff checks and formatting
• Python compilation of the affected modules
• git diff --check

The focused pytest suite was not run locally because the repository's locked runtime supports Linux while the available development host is macOS ARM. The tests should be run in Linux CI or on the cluster.

Base branch dependency: amahishi/prefix-recovery-core.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[macandro96]

/ok to test 2c646aa

[macandro96]

/ok to test 9812a07

[macandro96]

Superseded by #4508, which carries the batched generation-prefix writes (plus coalesced restore reads) on the current prefix-recovery stack. Closing for now.