Skip to content

feat: parallel TTS prefetch + math-to-speech prompt rules - #144

Merged
elimelt merged 1 commit into
mainfrom
feat/tts-parallel-prefetch
Aug 2, 2026
Merged

feat: parallel TTS prefetch + math-to-speech prompt rules#144
elimelt merged 1 commit into
mainfrom
feat/tts-parallel-prefetch

Conversation

@elimelt

@elimelt elimelt commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Summary

Two improvements to the TTS reader, following on from the serial queue +
constrained rewriting work in #143.

1. Smoother playback — parallel prefetch queue

The old queue synthesized one chunk at a time and stayed only 2 chunks
ahead, so audio stalled whenever playback caught up to the single
in-flight LLM+TTS pipeline.

  • CONCURRENCY = 4 synthesis pipelines now run in parallel, each
    claiming the next un-started index via a next pointer.
  • LOOKAHEAD = 6 keeps a deep buffer ahead of playback.
  • Exactly-once is preserved: const i = next++ claims a distinct
    index per worker with no await in between (JS is single-threaded).
  • In-order playback is unchanged: awaitChunk(i) still waits for
    index i regardless of the order results land.
  • Epoch cancellation, waiter wake-ups, and object-URL cleanup all
    preserved; stop() resets next.

Safe now that the upstream rate limit was raised ~10x. Lower
CONCURRENCY if 429s reappear.

2. Better math-to-speech prompt

LLM_PROMPT now lists explicit conversion rules instead of relying on a
couple of examples:

  • Subscripts: x_i -> "x i"
  • Powers: x^2 -> "x squared", 2^n -> "two to the n"
  • Function application: f(x) -> "f of x", g(x, y) -> "g of x and y"
  • Big-O, fractions, relational/arithmetic operators
  • Common LaTeX commands (\sqrt, \sum, \approx, \to, ...)
  • Greek letters read by name; code identifiers with separators spoken as
    spaces

Plus an anti-refusal directive so short technical fragments aren't
mistaken for instructions. CACHE_VERSION bumped to 3 so rewrites cached
under the old prompt are ignored.

Testing

  • Standalone simulation of the parallel queue: exactly-once production,
    concurrency capped at 4 (real parallelism observed), correct under
    variable per-chunk latency, clean stop/restart.
  • Live LLM check of the new prompt: all requested rules apply and plain
    prose is left verbatim. Remaining small-model flakiness (occasional
    refusal on a bare a != b) is caught by the existing length-ratio
    guard, which falls back to reading the raw text.
  • npm run build clean (1700 files); new logic confirmed in the shipped
    bundle.

Smoother playback and more faithful math narration for the TTS reader.

Playback: replace the single serial synthesis worker with a parallel
prefetch pool (CONCURRENCY=4) and a deeper look-ahead (LOOKAHEAD=6). Up
to four LLM+TTS pipelines run at once, each claiming the next un-started
index via a `next` pointer, so the buffer stays well ahead of playback
and audio no longer stalls between chunks. Exactly-once production and
strict in-order consumption are preserved (index is claimed atomically
with `next++`; playback still waits on `awaitChunk`), along with epoch
cancellation and object-URL cleanup. Safe now that the API rate limit
was raised; lower CONCURRENCY if 429s return.

Prompt: rewrite LLM_PROMPT with an explicit math-to-speech rules list —
subscripts (x_i -> "x i"), powers (x^2 -> "x squared", 2^n -> "two to
the n"), function application (f(x) -> "f of x", g(x,y) -> "g of x and
y"), Big-O, fractions, relational/arithmetic operators, common LaTeX
commands, Greek letters by name, and code identifiers. Add an
anti-refusal directive so short fragments are treated as content, not
instructions. Bump CACHE_VERSION to 3 to invalidate rewrites cached
under the old prompt.
@elimelt
elimelt merged commit e6e8904 into main Aug 2, 2026
1 check passed
@elimelt
elimelt deleted the feat/tts-parallel-prefetch branch August 2, 2026 05:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant