Skip to content

Voice: a dot-matrix glyph on every surface, phone dictation through the host - #916

Draft
gaelcado wants to merge 8 commits into
zeronsh:mainfrom
gaelcado:feat/voice-glyphs
Draft

gaelcado wants to merge 8 commits into
zeronsh:mainfrom
gaelcado:feat/voice-glyphs

Conversation

@gaelcado

@gaelcado gaelcado commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

One dot-matrix voice glyph for every surface, on desktop and iPhone, replacing the orb, plus dictation from the phone through the chat's host.

  • Shared glyph engine (zeron-orb::glyph): a field (the call's disc) and a beam (a strip) decide every dot's level, colour and size from the two voices' bands and the call's mode, on both platforms. Light is a halftone: dots swell and touch as they light. Each dot takes one of the working glyph's three inks (GSPIN_ROW_TINTS), scattered at random and never blended; the user's voice takes the accent. Modes pass into each other through a layered passage so a change mid-fade joins the blend. The old orb engine, its presets and the thinking-orbs chain (and its licence) are gone.
  • Desktop: the stage orb in the composer's own disc, graded from a hot heart to small dots at the rim, lit under the pointer; a call tray above every conversation's composer on the queue's surface, its beam the dictation track's twin; the beam in miniature in the sidebar footer. The user's voice is read from the microphone's own spectrum (zeron-voice::spectrum), gated while Codex plays so playback does not come back as the user. Continuous loud speech no longer pins the glyph: the loudness scale runs to full scale and the heart beats with the syllables.
  • iOS: GlyphView paints the engine's frames through the mobile FFI for the stage, the tab accessory and the bar item; Live Activity assets follow. The stage anchors the orb under the host name and gives the words the whole band to the controls.
  • Transcript (both platforms): every turn of the call is kept and a turn's whole text with it, centred in its band while it fits and scrolling once it does not, the newest in view. The transport's 32 KB per-message guard is unchanged; the caption's own ceiling is 1 MiB per turn and keeps the beginning.
  • Captions: Codex's item/started carries the role before any delta, so a turn is attributed as it begins; one still unnamed after a short grace is the user's, and the provider's word wins.
  • Dictation from the phone: no model on the phone. It captures, resamples to 16 kHz PCM and streams over the device relay to the chat's host, whose own Parakeet worker (zeron-voice, fed from the network) writes it down and sends partials and the final back. Tap to start, tap the stop to transcribe. The host advertises dictation-host-v1; the RPC surface is StartDictationV1, FeedDictationV1, FinishDictationV1, CancelDictationV1 and the OwnDictationV1 stream. One dictation at a time per host, shared with its own composer; an unfinished one is dropped after 90 s.

Look closely at

  • Compatibility: VoiceEvent::Partial gains an optional role (#[serde(default)]); dictation is gated on the dictation-host-v1 capability, and a phone talking to an older host gets a toast instead of a failure. No persisted keys or RPC names change.
  • crates/voice-session keeps every caption turn now (was 64) and no longer restarts a turn past 32 KB. The mobile state carries all turns on every update; fine for hundreds of turns, unmeasured beyond.
  • The desktop reads the microphone a second time for the glyph (zeron-voice::spectrum::Listener), levels only, only while a call is live and unmuted; ZERON_VOICE_SPECTRUM=0 turns it off.
  • New dependencies: zeron-mobile gains base64 and uuid (already in the workspace) for the dictation FFI; zeron-engine depends on zeron-voice.

Device deletion (a deleteDevice mutate op) was developed alongside and is split out for its own PR.

Test plan

  • cargo test --locked -p zeron-ui --lib -- --test-threads=1 (1776 passed)
  • cargo test --locked -p zeron-engine --lib
  • cargo test --locked -p zeron-harness: 319 passed; two timing-bounded tests (executable::tests::version_probe_bounds_hangs_and_rejects_nonzero, opencode::tests::stalled_prompt_post_has_a_bounded_timeout) failed once while the other suites ran alongside and pass in isolation; code untouched by this PR
  • cargo test --locked -p zeron-sync --lib
  • cargo test --locked -p zeron-preview: not completed here; its tests/leak integration test sat idle for 15 minutes on this machine and was killed (crate untouched by this PR)
  • cargo test --locked -p zeron-orb -p zeron-mobile -p zeron-voice -p zeron-voice-session -p zeron-proto -p zeron-doc -p zeron-rpc
  • cargo test --locked -p zeron-engine --test workspace_sync
  • cargo clippy --locked --all-targets on the touched crates: no new warnings
  • New tests: glyph engine (pointer touch; a mode arriving mid-passage joins the blend without a jump), caption (every turn kept; a UTF-8 turn past 32 KB keeps its first and last words), workspace capability fixture, iOS DictationDraftTests
  • Manual, macOS dev build: stage, call tray, dictation track, footer in dark and light, frosted; stage transcript centred with 1–4 turns then scrolling with a top fade; pointer over the orb
  • Manual, iPhone 13 Pro dev build and simulator: stage in dark and light, transcript centred then scrolling, tab accessory and bar glyph, tap-to-dictate UI over a scripted transcriber
  • Not done: a live phone-to-host dictation against a host with the model; a live call judged for fast-speech reactivity (the preview feeds a synthetic wave); Live Activity on a real lock screen

Screenshots

Desktop stage, call strip and dictation track in dark and light; iPhone stage in dark and light. Attached below.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

zeron-orb now holds one glyph engine for every platform: a field (the
call's disc) and a beam (a strip), each deciding every dot's level,
colour and size from the two voices' bands and the call's mode. Light is
a halftone: dots swell and touch as they light, with one of the palette's
three inks scattered at random per dot and never blended. Modes pass into
each other through a layered passage, so a change mid-fade joins the
blend. A pointer resting on a field lights the dots under it.
…the footer

The stage orb is the glyph's field in the composer's own disc, graded
from a hot heart to small dots at the rim; the pointer over it lights the
dots under it. Every conversation's composer carries the call in a tray
on the queue's surface, its beam the dictation track's twin. The sidebar
footer shows the beam in miniature. The user's voice is heard through the
microphone's own spectrum, gated while Codex plays so it does not come
back as the user's. The stage transcript keeps every turn, centred while
it fits and scrolling once it does not.
GlyphView paints the frames zeron-orb's glyph engine decides, through the
mobile FFI, for the stage, the tab accessory and the bar item; the Live
Activity's assets follow. The stage anchors the orb under the host name
and gives the words the whole band to the controls, centred while they
fit and scrolling after. The thinking-orbs chain and its licence go.
A host advertising dictation-host-v1 takes a phone's 16 kHz PCM over the
device relay (Start/Feed/Finish/Cancel, OwnDictation events), feeds it to
its own Parakeet worker and streams partials and the final back. The host
takes one dictation at a time, shares admission with its own composer and
drops one nobody finished after 90 seconds. No speech leaves the user's
machines.
The composer's microphone captures, resamples and streams to the chat's
host; words land in the draft as partials and settle with the final. Tap
to start, tap the stop to transcribe; no hold-to-talk on the phone. The
track over the controls is the glyph's beam with the recording clock.
Silence, denial, failure, cancellation and leaving keep the draft.
Codex's item/started carries the role before any delta, so a turn is
attributed as it begins; one still unnamed after a short grace is the
user's, and the provider's word wins when it comes. The caption keeps
every turn of the call and a turn's whole text, bounded only by a 1 MiB
runaway ceiling that keeps the beginning.
The engine now hosts phone dictation on zeron-voice, whose cpal capture
links ALSA on Linux, so the core job installs its headers. The orb's
geometry integration test went with the orb; the glyph engine's tests run
in voice-tests (cargo test -p zeron-orb).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant