Skip to content

perf: reduce Compact output tokens with positional JSON - #439

Open
lahfir wants to merge 8 commits into
laf/compact-generationfrom
laf/optimize-compact-wire-tokens
Open

lahfir wants to merge 8 commits into
laf/compact-generationfrom
laf/optimize-compact-wire-tokens

Conversation

@lahfir

@lahfir lahfir commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor
  1. Compact generation spent output tokens on repeated field names and empty fields, and its prompt repeated the catalog that the schema already describes.
  2. Props now stream as positional tuples with their order described in the schema, which compared with feat: add compact JSON generation and playground mode #438 uses about 60% fewer output tokens per element, about 80% less input, and half the generation time.
  3. Builds on feat: add compact JSON generation and playground mode #438.

Replace the two wire profiles with one positional JSON format. Keep state,
expressions, validation, and incremental publication while removing repeated
field names and unused trailing values.
@lahfir
lahfir added this pull request to stack #440 October 27, 2026 22:57
@vercel

vercel Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
json-render Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-chat-demo Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-dashboard-demo Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-game-engine-demo Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-image-demo Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-no-ai Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-react-email-demo Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-react-pdf-demo Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-react-three-fiber-demo Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-remotion-demo Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-solid-demo Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-svelte-chat-demo Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-svelte-demo Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC
json-render-vue Ready Ready Preview, v0 Nov 3, 2026 6:06am UTC

@lahfir lahfir changed the title Optimize Compact with catalog-derived tuples perf: reduce Compact output tokens with positional JSON Oct 27, 2026
Comment thread packages/core/src/compact-wire.ts Resolved
bits-ui resets body styles 24ms after a dialog closes. The Dialog test was
the last in its file, so CI sometimes tore down jsdom before that timer ran
and reported an unhandled 'document is not defined' error. Unmount and wait
for the timer before the test ends.
The Compact prompt repeated each component's description and prop order next
to a schema that already carried the descriptions. Each props tuple now
describes its positions ("Props in order: label, value, detail, trend."),
component descriptions list their events, and the prompt keeps only the wire
rules and actions.

On the comparison catalog this cut input by about 1,700 tokens per request
(28%). Ten runs per setup across five prompts were all valid with no swapped
props. The test encoder now reads prop order and slots from the schema.

The descriptions sit on the tuple definitions, not beside $refs. A title next
to a $ref made the provider inline the recursive schema and raised input from
about 3K to 283K tokens.
Compact generation needs a provider that enforces bounded tuples and
recursive schemas. Anthropic rejects the schema, and without enforcement
models break the wire format's records, metadata and value wrappers.

createCompactGeneration(catalog, { strict: false }) now returns a flat
[depth,type,props,meta?] format with plain JSON values, a prompt that
lists the catalog, a loose tool schema, and a decoder that validates each
element against the catalog as it streams. It emits the same JSONL
patches, so renderers are unchanged. Strict mode stays the default; both
modes now share the stream compiler and element validation.

Tested on Claude Haiku 5.5 and Sonnet 5.5 over 10 prompts each: 8/10 and
10/10 valid (JSONL: 8/10 and 10/10), with about half the output tokens
per element and less than half the input. Named slots are not supported
in relaxed mode.
@vercel-internal-security-reviewer

Copy link
Copy Markdown

Security review details

Comment thread packages/core/src/compact-generation.ts Resolved
Comment thread packages/core/src/compact-generation.ts Resolved
- Relaxed meta now accepts only visible, on, watch and repeat (plus prop
  names, which move into props). Any other key, such as children or
  slots, is rejected, and compiler-owned type, props and children are set
  after metadata, so model output cannot inject child references.
- Relaxed elements must have 3 or 4 positions; longer arrays used to
  pass and silently drop the extra data.
- createCompactGeneration now returns the exported CompactGeneration
  type, so consumers no longer infer a type that points into zod
  internals (TS2742 in apps/web).

Tested with core type-check, the core test suite, and the web app
type-check after building core.

This branch was successfully deployed

14 active deployments
Preview – json-render-svelte-chat-demo — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-chat-demo — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-game-engine-demo — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-no-ai — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-react-three-fiber-demo — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-dashboard-demo — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-react-pdf-demo — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-remotion-demo — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-image-demo — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-react-email-demo — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-solid-demo — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-svelte-demo — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Preview – json-render-vue — 8cf26909 Deployed Nov 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants