[hsliuustc0106]

Required human contributor answers

Answer every question before implementation and PR submission, including drafts.
Reuse explicit, still-applicable human answers with a traceable source reference;
ask the human for anything missing and wait. An agent must not invent answers.
Blank fields, placeholders, or an agent-written rationale do not satisfy this gate.
Share only information and source links appropriate for this PR's audience.

1. What outcome are you trying to achieve?

   Answer: Remove the in-request prompt-scratch grow on short-to-first-long transitions for StartLux Instant, so a later Public231 C1 remeasure can show pooled p50 and pooled p95 both improving and the 2048+ p95 dropping, with no quality collapse. Source: tip2 task. Owner Hongsheng approved tip2 in the system1 room (InferRSI spec OK; owner said proceed without waiting).

2. What is the highest-leverage variable for that outcome?

   Answer: Device scratch capacity at worker startup. On-demand growth to the next multiple of 1024 tokens syncs the stream inside the first prompt that exceeds the current cap. The task cites a profile of about +35 ms on the first long prompt after short ones. Process warmup is one short choice, and bench warmup can stay on short rows. Source: tip2 task.

3. How will this work move that variable rather than create busywork?

   Answer: Reserve that scratch once at startup for the serve limit (STARTLUX_MAX_MODEL_LEN), using the existing 1024-token grow quantum, and keep on-demand grow for a forward longer than the reservation. The allocation and its stream sync leave the measured request path. Pack-V (tip1) was dropped and did not reduce 2048+ p95. Source: tip2 task and the system1-room approval above.

4. Who is the customer or user, and what do they actually need?

   Answer: Operators serving StartLux Instant on the Public231 C1 workload, who need the long-prompt tail to come down without a short-bucket regression. Source: tip2 task. Owner Hongsheng approved tip2 in the system1 room and said proceed without waiting.

After answering the four questions: is a PR needed at all to achieve this outcome?

Human decision and rationale (or explicit source reference): Yes. Owner Hongsheng approved tip2 in the system1 room (InferRSI spec OK) and said proceed without waiting. The reservation has to land on a branch so the same Public231 C1 protocol can remeasure it. That approval is to proceed with the tip. It is not a merge yes. This pull request stays draft.

For a bug fix: link actual observed failure evidence.
Include the observed result of a minimal reproduction, failing regression test,
or relevant trace. Label untested diagnoses as unverified. For a non-bugfix,
state why this is not applicable.

Evidence: Not applicable. This is a latency tip, not a bug fix. The about +35 ms profile figure is the task's citation. This change did not remeasure it.

Purpose

Tip2 only. RSI phase: launch/post. Primary change: at StartLux Instant worker startup, reserve prompt scratch for the serve limit so the first long forward does not pay the on-demand realloc and stream sync. Alternatives considered and left out of this tip: pack-V (tip1, commit 56d4cc4c, PR #178 closed), Ignition, CUDA graphs, cuBLASLt plan-cache, and kernel rebinds.

Negative lesson: tip1 pack-V alone did not reduce 2048+ p95.

The pre-size ceiling is STARTLUX_MAX_MODEL_LEN (default 65536, the vLLM release max-model-len; allowed range 1..=262144). The worker rounds that serve limit up with the existing grow quantum, 1024 tokens. A Public231 longest-sample length is not hardcoded. Lowering or raising STARTLUX_MAX_MODEL_LEN changes both the 422 cap and this reservation.

On-demand grow stays in prepare_scratch. A forward longer than the reserved cap still reallocates and syncs the stream. Prompts over the serve limit still return 422 and do not allocate.

Git base for this branch: a41e379453d4a8dc3cbe3f76c8754753c42230a1 (main).

VRAM cost (formula, not a device reading)

The count is the default-kernel scratch DeviceBuffer: the region list in scratch_regions (the same offsets Scratch::new allocates) plus the default GDN workspace mirrored from cs1_gdn_workspace_floats (gdn_prefill.cu Layout: chunk 64, head dims 128, 256-byte alignment, (bytes + 3) / 4 floats). Release builds still pass the CUDA symbol into the allocator. Weights, LoRA work, and the 32 MiB GEMM workspace are outside this figure.

Serve limit	Reserved cap	Scratch bytes	MiB (bytes / 1024^2)
65536 (default)	65536	16077160448	15332.375
1024 (one grow quantum)	1024	251205632	239.568359375
262144 (native context)	262144	64308641792	61329.500

Extra at the default, versus the 1024-token scratch a short warmup would allocate by itself: 15825954816 bytes (15092.806640625 MiB). The startup log prints the serve limit, the reserved cap, the allocated byte count, and that extra. If the device buffer disagrees with the formula, the log prints both sizes.

Test Plan

CPU only. No GPU. Rustc 1.99.1.

cargo fmt --all -- --check
cargo test --locked -p omni-startlux-native --test contract
cargo test --locked -p omni-qwen3-5-native --lib prompt_scratch_quantum
cargo clippy --locked -p omni-qwen3-5-native --lib --tests -- -D warnings
cargo clippy --locked -p omni-startlux-native --all-targets -- -D warnings

The contract test locks the ceiling to the serve limit (1 and 1000 -> 1024 tokens, 1025 -> 2048, 3000 -> 3072, 8192 -> 8192, default 65536 -> 65536 and 16077160448 bytes, native context 262144 -> 64308641792 bytes) and rejects 0 and 262145. The Qwen unit test locks the GDN workspace mirror and the scratch-region byte count, including that the byte function does not quantize by itself.

System1-Omni Version / Commit: tip 95444eb042efb2bd5a6f83330ef66f56745deb30 on a41e379453d4a8dc3cbe3f76c8754753c42230a1.

Test Result

All five commands above passed. cargo test --locked -p omni-startlux-native --test contract: 9 passed. prompt_scratch_quantum_and_default_bytes_match_the_allocator: passed.

Not run: workspace cargo test / cargo clippy CI suite, CUDA contract job, GPU numerical parity, and Public231 C1. No device was available for those.

Demo / evidence

No latency claim. This tip has not been remeasured.

Measured baseline to beat (not this branch's git base):

• runtime 51f80ad681c8f7b1be9b22b5e700bef34e33534c
• Public231 C1 pooled p50 / p95: 7.338 / 51.816 ms
• pass rate 693/693
• warmup 5, concurrency 1, 3 rounds
• device: L20X (nvidia-smi)
• compute capability: 8.9
• thinking treatment: enabled: false
• clock: post_to_body_json_and_schema_validation
• 2048+ bucket: n=108, p50 50.346 ms, p95 71.942 ms

Gate for a later remeasure of this tip SHA, same protocol, same recorded device and compute capability: pooled p50 and pooled p95 both improve, 2048+ p95 drops, and quality does not collapse. Until that remeasure, the latency result is unmeasured. Do not treat the profile citation or this formula as a serving speedup.

Memory figures above are the scratch formula. L20X free memory was not read. Video: N/A, no UI change. Output parity: unverified, no GPU forward was run. The change does not alter letter readout, temperatures, or the 422 contract; contract tests cover the CPU decision path.

Self-review

Before marking this PR ready for review or requesting maintainer review, complete
the self-review checklist.
Keep the PR in draft while this work is incomplete.
For agent assistance, use the optional precheck-pr skill.

• I have reviewed the full diff and addressed the issues I found.
• I have checked that the change follows the project's architecture and stays focused on the stated purpose.
• I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
• I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

Draft until a separate owner yes to mark ready or merge. Passing CI is not that yes. A remeasure is still required before any latency verdict. Diff is about 399 insertions and 62 deletions across 10 files, under the 3000-line authored-code threshold. Scope is StartLux startup scratch reservation plus the shared scratch size helper it calls. Pack-V, Ignition, CUDA graphs, plan-cache, and kernel rebinds are absent.