Budget-checked encode entry points (try_encode_bounded) — check a caller size cap without a second size pass

[iainmcgin]

Motivation

Writers commonly sit in front of a transport or store with a hard payload cap far below the protobuf limit — an RPC frame limit, a message-queue cap, a storage row limit. The pre-flight question they need answered is "will this message fit my budget?", ideally before any bytes are produced.

#271 establishes the floor for this: generated compute_size now accumulates in u64 and saturates (saturate_size), every encode entry point checks the result against MAX_MESSAGE_BYTES, and the try_* twins return EncodeError::MessageTooLarge instead of the historical silent u32 wrap past 4 GiB. But that check is fixed at the protobuf 2 GiB limit; a caller's own budget isn't expressible.

What a caller can write once #271 lands:

let len = pool.try_encoded_len(&msg)?;     // size pass 1: full tree walk
if len > FRAME_BUDGET {
    return Err(Frame::TooLarge(len));
}
pool.encode(&msg, &mut buf);               // size pass 2: the same walk again

Correct, but the size pass runs twice — and avoiding repeated size computation is exactly what SizeCache exists for: after compute_size, the cache holds everything write_to needs. The check-then-encode flow just has no public entry point that exploits it: try_encoded_len discards the populated cache, and driving compute_size / write_to directly means taking on the traversal-order invariant and the checked_encode_size funnel by hand, which the docs (reasonably) discourage.

Proposed API

Budget-checked encode entry points, following the crate's naming grid (encode panics, try_encode returns Result — a bounded encode is inherently fallible, so it takes the try_ prefix):

impl SizeCachePool {
    /// Compute the encoded length (one size pass). If it exceeds
    /// `min(max_bytes, MAX_MESSAGE_BYTES)`, return an error having written
    /// nothing; otherwise encode — reusing the already-populated cache —
    /// and return the length.
    pub fn try_encode_bounded<M: Message>(
        &mut self,
        msg: &M,
        max_bytes: u32,
        buf: &mut impl BufMut,
    ) -> Result<u32, EncodeError>;

    /// `ViewEncode` counterpart.
    pub fn try_encode_view_bounded<'a, V: ViewEncode<'a>>(
        &mut self,
        view: &V,
        max_bytes: u32,
        buf: &mut impl BufMut,
    ) -> Result<u32, EncodeError>;
}

The implementation home is a provided try_encode_bounded_with_cache on Message (and the ViewEncode counterpart), sitting next to the existing try_encode_with_cache — the same two-pass body with the budget comparison inserted between checked_encode_size and write_to. The pool methods wrap it with acquire/release, like every other pooled entry point. No new invariants: both passes take the message by &, and the cache stays private to the call. On Err, nothing is written and the spill buffer still returns to the pool.

Since the provided method carries the whole body, the pool-less twin is one trivial declaration and is included for grid symmetry:

// Message (and ViewEncode analogously):
fn try_encode_bounded(&self, max_bytes: u32, buf: &mut impl BufMut) -> Result<u32, EncodeError>;

Returning the length on success costs nothing and is useful for metrics and framing.

Error shape

A dedicated variant, not a reuse of MessageTooLarge:

#[non_exhaustive]
pub enum EncodeError {
    MessageTooLarge,
    /// The encoded size exceeds the caller-supplied budget passed to a
    /// `try_encode_bounded` entry point (but is within the protobuf limit).
    ExceedsBudget { len: u32, max_bytes: u32 },
}

EncodeError is already #[non_exhaustive], so the addition is semver-safe. Reusing MessageTooLarge would be misleading: its documented meaning is "no conforming protobuf decoder will accept these bytes — shrink or split the message", which does not describe a caller-budget violation.

Precedence is deterministic: a size over MAX_MESSAGE_BYTES returns MessageTooLarge (preserving #271's property that no exact size is promised once compute_size saturates); otherwise a size over max_bytes returns ExceedsBudget, where len is exact — a message within the protobuf limit always has a representable size.

max_bytes is u32 to match the size domain throughout the crate; callers holding a usize transport budget cast at the boundary (u32::try_from(budget) or clamp).

Scope note

For "check now, encode later (or never)", try_encoded_len already covers it once #271 lands; this proposal is specifically about not paying the tree walk twice when the check is immediately followed by an encode.

Alternatives considered

• Status quo (try_encoded_len + compare + encode): works, costs a second full tree walk. How much that matters varies with message shape — the size pass is a full tree traversal, so the relative cost is measurable with the existing bench suite for any workload where it's in doubt.
• Manual compute_size / write_to: public and sufficient, but the caller inherits the traversal-order invariant and must run the checked_encode_size funnel themselves — effectively re-implementing the entry points the crate already centralizes.
• Prepared-encode guard (pool.prepare(&msg)? → PreparedEncode exposing .encoded_len() then .write(buf)): more flexible when the budget decision happens far from the encode site, but adds a public type plus cache-return plumbing. Could be layered later; try_encode_bounded covers the common case with no new types.

[iainmcgin]

[claude code] Fixed by #320 (f3d1abd), shipping in 0.9.0 — thanks to @rsd-darshan for the implementation.

Message::try_encode_bounded(max_bytes, buf) and ViewEncode::try_encode_bounded, plus *_with_cache and SizeCachePool equivalents. One size pass, budget check before write_to, nothing written on Err (including bytes a caller had already framed into the sink), and the encoded length returned on Ok. New EncodeError::ExceedsBudget { len, max_bytes }, with MessageTooLarge taking precedence when both limits are exceeded.

This is exactly the flow the issue asked for: the check-then-encode pattern now costs one tree walk instead of two, because the populated SizeCache carries over to the write.