Skip to content

feat: speak display math + switch TTS rewrite to gemma4:e4b - #148

Merged
elimelt merged 1 commit into
mainfrom
feat/tts-display-math
Aug 2, 2026
Merged

feat: speak display math + switch TTS rewrite to gemma4:e4b#148
elimelt merged 1 commit into
mainfrom
feat/tts-display-math

Conversation

@elimelt

@elimelt elimelt commented Aug 2, 2026

Copy link
Copy Markdown
Owner

What

Two changes to the TTS reader:

  1. Speak display math. Quartz emits $$...$$ as a <span class="katex-display"> that is a sibling of paragraphs, not inside one, so the block selector (p, li, dt, dd, h1-h6) never collected it. Verified across the built site: all 402 katex-display spans sat outside any block element — every standalone equation on every note was silently dropped from the reading. Fix: add .katex-display to BLOCKS. The existing nested-block filter and the data-tex path in textOf handle the rest; each display equation becomes its own chunk carrying its full LaTeX. This surfaced 533 pure-math blocks corpus-wide that were previously inaudible.

  2. Switch the rewrite model to gemma4:e4b (think: false). Eval vs the shipped qwen2.5-coder:7b (both with the S3 prompt):

gemma4:e4b qwen2.5-coder:7b gpt-oss:20b (low) gemma2:2b
Semantic suite 18/18 18/18 18/18 11/18
Residual LaTeX (15 real blocks) 0/15 1/15 0/15
Warm rewrite ~2.5s 3.4–4.0s 7.8s 8.7s
Cold prompt eval (~1.6k tok) 28s 49s
Timeline wall (120s audio) 122.4s 127.7s

CACHE_VERSION bumped to 6 so stale qwen rewrites regenerate.

Sample readings (display equations, gemma4:e4b)

  • t_h \leq t_{input,i} \leq T_{clk} - t_s ~ \forall i → "t h is less than or equal to t input i is less than or equal to T clock minus t s for all i"
  • T = \frac{80 \times 10^9 bytes}{3.35 \times 10^{12} bytes/s} \approx 23.9 ms → "T equals eighty times ten to the nine bytes over three point three five times ten to the twelve bytes per second approximately twenty three point nine milliseconds"

Verification

  • Timeline replay on a display-math-heavy note (static-timing-analysis, 24 chunks, 11 LLM-bound after the fix), warm cache: 0.7s to first audio, one 1.6s stall, wall 188.9s for 186.5s of audio — near-realtime with double the previous LLM load.
  • Cold-cache replay shows one 13s stall from the one-time prompt eval; in the browser the urgency-skip path covers this (deterministic rendering plays immediately while the quality rewrite fills the cache).
  • Build passes; emitted bundle parses and contains the new selector, model, think: false, and cache v6.

Display math was silently skipped: Quartz emits $$...$$ as a
<span class="katex-display"> that is a sibling of paragraphs, not
inside one, so the block selector (p, li, dt, dd, h1-h6) never
collected it. Verified across the built site: all 402 katex-display
spans sat outside any block element, so every standalone equation was
dropped from the reading. Fix: add .katex-display to BLOCKS; the
existing nested-block filter and the data-tex path in textOf handle
the rest, so each display equation becomes its own chunk carrying its
full LaTeX (533 pure-math blocks surfaced corpus-wide).

Also switches the rewrite model from qwen2.5-coder:7b to gemma4:e4b
with think: false, per eval:

- semantic suite: 18/18 both, but gemma has 0/15 residual LaTeX on
  real blocks vs qwen's 1/15
- warm rewrites ~2.5s vs 3.4-4.0s; cold prompt eval 28s vs 49s
- timeline replay (stable-matching, warm): wall 122.4s for 120.2s of
  audio, llm avg 5.4s max 6.8s (vs qwen 127.7s wall, max 18.5s)

gpt-oss:20b matched semantically but ran 2-3x slower; gemma2:2b
failed the suite (11/18) and was rejected.

CACHE_VERSION bumped to 6 so stale qwen rewrites regenerate.

Verified on a display-math-heavy note (static-timing-analysis, 24
chunks, 11 LLM-bound after the fix): warm replay 0.7s to first audio,
one 1.6s stall, wall 188.9s for 186.5s of audio.
@elimelt
elimelt merged commit 976d57d into main Aug 2, 2026
1 check passed
@elimelt
elimelt deleted the feat/tts-display-math branch August 2, 2026 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant