Repository navigation
feat: Hex 2.2.0 — faster dictation and the Meetings beta - #133
Merged
Merged
Conversation
GGUF sessions on GPU devices now request one CPU thread instead of
transcribe.cpp's default min(8, ncpu). Parakeet's per-token TDT decoder
runs on CPU even with Metal encoding, and ggml's spin-waiting worker pool
stalls it under CPU load. CPU-only devices (Linux Backend::Auto fallback)
keep the library default so the encoder is not serialized.
The macOS inference thread moves from user-initiated to user-interactive
QoS. ggml creates workers with plain pthread_create, which starts them at
QOS_CLASS_DEFAULT rather than inheriting (confirmed with a probe); with one
thread no workers are created and decoding stays on this thread.
transcribe-cpp 0.2.4 bench, Unified English Q8, Metal, Mac Studio (18
cores, already loaded by other work), user-interactive QoS, median ms for
3/30/90 s clips:
+ 36 `yes` procs: 1 thread 170/703/1819, 2 threads 136/1907/4940,
4 threads 178/4304/13519
no extra load: 1 thread 83/364/1631, 2 threads 64/705/2889
Not verified physically on the MacBook Pro or on Linux.
Clipboard managers such as Maccy and Raycast recorded every dictation because the paste write carried only plain text. Publish the transcript as one pasteboard item that also carries org.nspasteboard.TransientType and org.nspasteboard.AutoGeneratedType so conforming managers skip it. Generation-safe restoration of the previous clipboard is unchanged. Refs kitlangton/Hex#223
On "Dvorak – QWERTY ⌘" the plain layer puts V on keycode 47, but with Command held that key is ⌘. — so synthetic ⌘V posted ⌘. and paste beeped. Build a second snapshot map with UCKeyTranslate's cmdKey modifier state during initialize_layout and resolve Command shortcuts against it first, falling back to the plain snapshot. Headless lookups do the same. Refs kitlangton/Hex#162
Holding the shortcut without speaking made Cohere (and Whisper-style models) paste a hallucinated "thank you". Measure the loudest 30 ms window of the captured 16 kHz audio, DC removed and excluding model padding, and discard captures below -50 dBFS as empty: no inference, paste, History, or last-result update. The level and threshold are logged. The threshold is conservative so whispered speech is kept. Covers macOS hotkey, voice-delimited, and Voice Action captures and the Linux hotkey loop. Local API dictation and host-audio transcription deliberately keep returning raw model output. Refs #108
With Release when idle, the first hold after minutes of idle failed with the error tone. The cold stream reported "A buffer underrun or overrun occurred" — cpal 0.18's ErrorKind::Xrun from CoreAudio's processor overload listener, which does not stop the stream — and HEX treated it as fatal: the owner closed the device without retrying and recognition suspended the still-held gesture. - Log and ignore Xrun and RealtimeDenied stream errors; real sample gaps still fail the capture as discontinuities. - In release-when-idle mode, a stream failure before the hold threshold returns the press to pending and reopens, so release-before-ready discards and readiness captures from the original press. Later failures still fail the capture rather than paste a fragment. - Cold opens retry the selected microphone twice, 100 ms apart, before the existing automatic fallback. Refs #130
…into test/voice-perf-batch
…e-perf-batch # ------------------------ >8 ------------------------ # Do not modify or remove the line above. # Everything below it will be ignored. # # Conflicts: # src/dictation.rs
…ppression Stream each track through a 16 kHz resampler into pause-aware chunks (first pause after 20 s, hard cut at 30 s with a 2 s overlap) and transcribe them while recording on a utility-QoS session that waits for dictation capture and inference. Tracks that fall behind are rebuilt from their WAV after Stop. Seams drop repeated words, microphone utterances that echo overlapping remote speech are removed, and each finished meeting is written as Markdown to the configured folder. Remote speech is labeled Them.
…ding Meetings (beta) in Settings enables detection, the Meetings pane, and meeting paste in release builds, with a Screen Recording permission row and a transcript folder. Offers appear as a focus-free capsule with a countdown and stay snoozed for the call. The menu bar shows elapsed recording time with Stop & Transcribe, and a capsule asks Keep Recording or Stop & Transcribe when the call app leaves the microphone.
…Intelligence Add a weak-linked FoundationModels Swift bridge (availability, context size, guided-generation notes at utility priority) and a meeting notes worker that runs after final publication: bounded map/reduce chunking sized from the model context, a total deadline, manifest storage, and an in-place marker-delimited Markdown update that keeps the file name. Settings gains 'Summarize with Apple Intelligence' (default on) with a one-line availability; the Meetings pane and ready capsule show the generated title, notes, and a Regenerate Notes action. Debug CLI: hex meeting summarize <id|file>.
…eed detection Rolling chunks fell behind (packet-bounded queue) and the pane then showed the raw Moonshine draft without any echo suppression, so microphone twins of remote speech appeared as You. Coalesce rolling audio into one-second blocks, keep transcribed entries of a track that fell behind, judge the live draft with the same echo rules, split timed words into sentences, and classify microphone segments by comparing their energy with delayed system audio so misheard bleed is dropped and double-talk is kept.
…earch, rename, and delete List rows show the generated title, time, length, app, and a status chip, grouped by day. The detail leads with notes, then Them/You turns on opposite sides that follow the live tail, under one action row: Copy Notes, Copy Transcript, Reveal Markdown, Regenerate Notes, Rename, and a two-step Delete that moves the Markdown and recording to the Trash. Adds a deterministic meetings preview.
… summaries Name a recording from its opening (~90 s of speech or 600 words) with one title-only Apple Intelligence call, refreshed once at ten minutes, shown in the menu bar, Meetings pane, manifest, and exported file. After publication, write the final title and a summary of at most two sentences with one call over a bounded excerpt (opening, sampled middle, ending) instead of map-reduce. Remove action items, decisions, and chunking; older manifests still read. get_meeting_notes now returns the title and summary and points agents at the transcript.
One periodic rsync (scripts/sync-meetings-mirror.sh, LaunchAgent template) replaces an SSH session per MCP client, which exhausted the Pro's sshd. A copy of the binary named hex-meetings-mcp always runs hex mcp.
…te the instance-lock test Add a resampled delayed-release regression test (44.1/48 kHz, segmented and not) and an opt-in Unified English release-boundary harness. The instance lock test now uses a temporary directory instead of the real Application Support folder, which accumulated test-<pid>.lock files.
…nd add corpus harness
…/hex-voice-meetings # ------------------------ >8 ------------------------ # Do not modify or remove the line above. # Everything below it will be ignored. # # Conflicts: # docs/features/dictation.md
…out of Linux dead-code lints
kitlangton
marked this pull request as ready for review
October 16, 2026 16:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Long dictations were slow on a busy Mac: the Parakeet decoder ran 8 spin-waiting CPU threads, so under build load a 30 s dictation could take 12.8 s to transcribe, and nothing was transcribed until release. Meetings existed only as a debug prototype, with echo duplicating remote speech as
Youon speakers and no way for agents to read past calls. This PR ships both as Hex 2.2.0.What Changes
Faster dictation
Dictation accuracy
Fixes
Meetings (beta, opt-in)
~/Documents/Hex Meetings.hex mcp: a read-only stdio MCP server (list_meetings,search_transcripts,get_transcript,get_meeting_notes, Markdown resources). No writes, no network, no model calls.scripts/sync-meetings-mirror.shmirrors another Mac's meetings for local MCP clients.Scope
This branch combines
perf/decoder-threads,perf/incremental-transcription,fix/triage-batch,test/dictation-corpus, and the meetings stack. Semantic transcript search, calendar titles, and the native macOS shell (docs/plans/native-macos-app.md) are follow-ups. The version bump to 2.2.0 (build 20200) lands separately onmainas the release commit.Verification
cargo fmt --check cargo clippy --all-targets --all-features -- -D warnings cargo test git diff --checkcargo test: 584 passed, 13 ignored, plus the stdio MCP smoke test. Linux CI (testandnix) on this PR.scripts/dictation-corpus.sh).Youlines kept.hex mcpwas exercised over SSH from an OpenCode client: initialize 0.33 s,list_meetings18 ms,search_transcripts12 ms.