You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(mobile): resend dictation audio lost when the connection drops - #35777
When the phone's connection to the desktop drops while you're dictating, the audio it had just sent was lost, so a word or a second of speech went missing from the transcript. Now the phone sends that audio again once it reconnects, and the desktop makes sure each piece is heard once, in order.
What Changed
Before: since #35338, dictation keeps recording through a reconnect, but every chunk in flight on the dying connection was rejected as "delivery unknown" and dropped. That was about a second of speech on a hard drop. The phone couldn't resend because the host had no way to tell a resend from new audio, so a resend could have repeated words.
After:
Host (runtime-mobile-dictation-controller.ts, new mobile-dictation-chunk-order.ts): chunks carry a seq, and the host feeds them in seq order, each seq once. A chunk that arrives ahead of a missing resend is held until the resend fills the gap. The hold is capped at what the phone itself holds through a reconnect: 30s of 16 kHz PCM (the shared MOBILE_DICTATION_MAX_PENDING_AUDIO_BYTES) and 1,000 chunks (30s of 32ms native events). Past either cap, the oldest gaps are given up on until both hold again. Finish feeds whatever is still held. The speech.dictation.start reply now says dedupesResentChunks: true.
Phone (mobile-dictation-audio-chunk.ts, mobile-dictation-desktop-start.ts): every chunk carries a seq. Only when the start reply has the flag does the phone resend a delivery-unknown chunk, after each reconnect, while that recording is still the active one. It extends the resend fix(mobile): keep dictation recording through a reconnect and keep what was already said #35338 added for chunks stranded in a direct→Relay switch (sendAcrossConnectionMove) rather than adding a second path. A request that timed out on a connection still up is not resent there.
Why
Ordering has to live on the host. On a plain direct reconnect, chunks captured during the outage are parked in the transport's own connect wait and go out before any resend. On the OTA page they also cross the bridge. The phone's dictation code can't put a resend ahead of them, and an earlier version of this branch that tried was beaten by both routes in review. The host is the one place that sees every chunk, so it orders them there. Feeding a resend as soon as it lands, out of order, would garble the words around the drop.
The capability rides on the start reply, not a status-probe capability. It's free, it's answered by the exact host holding the session, and the phone already waits for that reply before it sends any audio.
Mixed versions behave as today:
Old phone, new host: no seq, so every chunk is fed directly.
New phone, old host: no flag, so nothing is resent. The old host's DictationChunk is a plain z.object, so it strips seq.
Not in this PR: noticing a silent drop quickly. On direct the liveness watchdog takes about 44s (20s idle, then 3 × 8s probes), which is past the host's 30s grace, and relay has no idle probe at all. So this PR doesn't change the silent-drop case. A stacked follow-up makes an overdue chunk reply fire a decisive liveness probe, so the drop is noticed within the grace on both the native app and the OTA page. The 30s grace is unchanged on purpose: raising it would hold the speech model and block other devices' starts for longer, only to cover a gap the follow-up closes.
Linked Issue
#35213: this fixes the lost speech on a hard drop or connection switch. Fast detection of a silent drop follows in a stacked PR.
Visual Proof
N/A: no UI changes. The change is in which audio reaches the host after a drop, and the tests below prove it through the real dictation hook and the real host controller.
Testing
Host (runtime-mobile-dictation-controller.test.ts): a resent chunk is fed once and in spoken order, and finish feeds what's still held. A gap that never fills is given up on at the cap instead of stalling the transcript. Both tests fail on the pre-fix code.
Phone (mobile-dictation-reconnect.test.tsx): through useMobileDictation, a chunk lost to a drop is resent to a host with the flag, and never to a host without it. A recording cancelled during the outage is not resent. Each fails with its guard removed.
Mobile RPC recording goldens for the dictation families replay clean. The two chunk goldens and the scenario were re-recorded for the new seq param.
Focused dictation suites (mobile hooks, platform capture, session dictation; host controller, orcad speech services, speech worker lifetime) pass, as do the changed-file code-quality and React Doctor gates. Full lint, typecheck and test are left to CI.
Platforms: logic is platform-independent (phone transport + Node host); not hand-tested on a device.
I manually tested these changes locally
Automated tests added/updated, or explained why not below
AI Disclosure
Review
Agent skill upstream boundary
Not applicable, or this change copies or mechanically translates no upstream skill-installer source, tests, fixtures, registry entries, path tables, comments, or documentation.
Notes
Ensure no issues in: Security, Cross-platform support (Linux, Windows, Mac), Remote SSH, Mobile, general backwards compatibility, performance
Security: the host hold is bounded by bytes (about 1.3 MB of base64) and by count (1,000 chunks), so a hostile seq or a flood of tiny chunks can't make it loop, throw or grow past that.
Remote / headless:orca serve and orcad hosts use the same controller.
Folder workspaces: not affected.
For blessing (new behaviour: a host capability, and what the phone resends):
Try: dictate on the phone over LAN, toggle the desktop's Wi-Fi off and on mid-sentence (a hard drop), keep talking, then stop. The transcript should include the words spoken right before and during the drop, with nothing repeated.
Mixed versions: an older desktop gets no resends (no flag in the start reply), and an older phone is fed as today (no seq).
Worktree for a dev build: /Users/kelvinamoaba/orca/workspaces/orca/dictation-dying-socket-audio (branch mobile-dictation-dying-socket).
Checklist
This PR is small and focused
I explained what changed and why (ELI5, the user-facing before/after, the mechanism, and why over the alternatives)
Before/after screenshots or videos attached for UI changes, or N/A with reason
Self-reviewed for correctness, security, and performance
Cross-platform, SSH/remote, and path/shortcut impact considered (or N/A)
pnpm lint, pnpm typecheck, pnpm test, and pnpm build pass (or CI will cover; local preferred)
The host feeds chunks in seq order, each seq once, holding audio that
arrives ahead of a resend for up to the reconnect grace, and says so in
its start reply. To such a host the phone resends chunks rejected as
delivery-unknown after each reconnect, so speech sent on a dying
connection is no longer lost.
Dictation chunk requests now include sequence numbers. The mobile client reads whether the host deduplicates resent chunks and retries eligible requests after reconnect while dictation remains active. The runtime buffers sequenced chunks, forwards them in sequence order, and drains remaining chunks before finishing. Updated scenarios, goldens, and tests cover sequence numbers, resend behavior, and chunk ordering.
Priority: Normal
Merge Risk: Low · up to 164bc
Resending lost dictation audio after a reconnect now works without an age cutoff. Two edge cases remain: in rare near-limit cases, a recoverable audio gap can be abandoned early, and if the connection never returns, stopping dictation can wait noticeably longer before failing. Neither is likely to cause broad breakage, but both are worth tightening.
Pre-merge checks: 4 · 1
Failed checks: 1 warning
Check name
Status
Explanation
Resolution
Docstring Coverage
Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta…
Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed checks: 4 passed
Check name
Status
Explanation
Title check
The title "fix(mobile): resend dictation audio lost when the connection drops" matches the main change. The PR adds sequence numbers to dictation chunks, has the phone resend chunks lost during a drop…
Description check
The description follows the repository template. It has ELI5, What Changed, Why, Linked Issue (#35213), Visual Proof (N/A with reason), Testing, AI Disclosure, Review, Agent skill upstream boundary, N…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title "fix(mobile): resend dictation audio lost when the connection drops" matches the main change. The PR adds sequence numbers to dictation chunks, has the phone resend chunks lost during a drop, and has the host reorder and deduplicate them. The title is specific and follows a conventional-commit style.
Full details: Description check
Explanation
The description follows the repository template. It has ELI5, What Changed, Why, Linked Issue (#35213), Visual Proof (N/A with reason), Testing, AI Disclosure, Review, Agent skill upstream boundary, Notes, and Checklist sections. It explains the before and after, the mechanism, mixed-version behavior, and the known limitation for silent drops. Some items are unchecked, including the manual testing box and the final lint, typecheck, test, and build checkbox, and the AI Disclosure and Review sections are empty. The linked issue is referenced as a plain issue number rather than a closing keyword, which the template expects as "Fixes #". These gaps are minor, so the description is mostly complete.
Full details: Docstring Coverage
Explanation
Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 4 functions across 9 files.
Resolution
Write docstrings for the functions missing them to satisfy the coverage threshold.
Finishing Touches: 💡 1
📝 Generate docstrings: 💡
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Caution
Some comments are outside the diff and can’t be posted inline due to GitHub limitations.
⚠️ Outside diff range comments (1)
Minor · Count decoded bytes for the holding limit. · mobile-dictation-chunk-order.ts:6
src/main/runtime/mobile-dictation-chunk-order.ts:6 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
Count decoded bytes for the holding limit.
Base64 padding makes the sum of chunk lengths exceed this limit before the decoded audio exceeds 960,000 bytes. (raw.githubusercontent.com)
In the new test, after releasing the ten small chunks, six 160,000-byte chunks remain. They fit exactly, but the new loop releases another chunk and skips the preceding gap. A later resend for that gap is then discarded.
Track each chunk’s decoded byte length instead of comparing the sum of encoded lengths.
Review info
Run configuration
Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Repository UI
Review profile: CHILL
Plan: Advanced
Run ID: 762da479-29f5-4f08-8cfc-971c16105e21
Commits
Reviewing files that changed from the base of the PR and between 06428c0 and 164bcc3.
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
Review info
Run configuration
Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Repository UI
Review profile: CHILL
Plan: Advanced
Run ID: 831ccb5b-b93a-431e-b991-8c9c21658b44
Commits
Reviewing files that changed from the base of the PR and between 164bcc3 and d873917.
Files selected for processing: 1
mobile/src/hooks/mobile-dictation-audio-chunk.ts
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
Review info
Run configuration
Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Repository UI
Review profile: CHILL
Plan: Advanced
Run ID: e8508a7b-b225-481e-b663-992dbbf40a54
Commits
Reviewing files that changed from the base of the PR and between d873917 and 164bcc3.
Files selected for processing: 1
mobile/src/hooks/mobile-dictation-audio-chunk.ts
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.
Fail the dictation when the chunk reconnect wait expires.
When a chunk's reconnect wait expires, sendAcrossConnectionMove rethrows the original delivery-unknown error. The chunk catch suppresses that error, so the dictation is not failed. finishDictation then waits for pending chunks, which adds up to 30 seconds. It then sends the finish request with a 75-second timeout, even though the host has already dropped the session.
Return a distinct reconnect-expired error from sendAcrossConnectionMove. Handle it in the chunk catch by calling failActiveDictation. The dictation can then end before finishDictation sends its request.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ELI5
When the phone's connection to the desktop drops while you're dictating, the audio it had just sent was lost, so a word or a second of speech went missing from the transcript. Now the phone sends that audio again once it reconnects, and the desktop makes sure each piece is heard once, in order.
What Changed
Before: since #35338, dictation keeps recording through a reconnect, but every chunk in flight on the dying connection was rejected as "delivery unknown" and dropped. That was about a second of speech on a hard drop. The phone couldn't resend because the host had no way to tell a resend from new audio, so a resend could have repeated words.
After:
runtime-mobile-dictation-controller.ts, newmobile-dictation-chunk-order.ts): chunks carry aseq, and the host feeds them in seq order, each seq once. A chunk that arrives ahead of a missing resend is held until the resend fills the gap. The hold is capped at what the phone itself holds through a reconnect: 30s of 16 kHz PCM (the sharedMOBILE_DICTATION_MAX_PENDING_AUDIO_BYTES) and 1,000 chunks (30s of 32ms native events). Past either cap, the oldest gaps are given up on until both hold again. Finish feeds whatever is still held. Thespeech.dictation.startreply now saysdedupesResentChunks: true.mobile-dictation-audio-chunk.ts,mobile-dictation-desktop-start.ts): every chunk carries aseq. Only when the start reply has the flag does the phone resend a delivery-unknown chunk, after each reconnect, while that recording is still the active one. It extends the resend fix(mobile): keep dictation recording through a reconnect and keep what was already said #35338 added for chunks stranded in a direct→Relay switch (sendAcrossConnectionMove) rather than adding a second path. A request that timed out on a connection still up is not resent there.Why
Ordering has to live on the host. On a plain direct reconnect, chunks captured during the outage are parked in the transport's own connect wait and go out before any resend. On the OTA page they also cross the bridge. The phone's dictation code can't put a resend ahead of them, and an earlier version of this branch that tried was beaten by both routes in review. The host is the one place that sees every chunk, so it orders them there. Feeding a resend as soon as it lands, out of order, would garble the words around the drop.
The capability rides on the start reply, not a status-probe capability. It's free, it's answered by the exact host holding the session, and the phone already waits for that reply before it sends any audio.
Mixed versions behave as today:
seq, so every chunk is fed directly.DictationChunkis a plainz.object, so it stripsseq.Not in this PR: noticing a silent drop quickly. On direct the liveness watchdog takes about 44s (20s idle, then 3 × 8s probes), which is past the host's 30s grace, and relay has no idle probe at all. So this PR doesn't change the silent-drop case. A stacked follow-up makes an overdue chunk reply fire a decisive liveness probe, so the drop is noticed within the grace on both the native app and the OTA page. The 30s grace is unchanged on purpose: raising it would hold the speech model and block other devices' starts for longer, only to cover a gap the follow-up closes.
Linked Issue
#35213: this fixes the lost speech on a hard drop or connection switch. Fast detection of a silent drop follows in a stacked PR.
Visual Proof
N/A: no UI changes. The change is in which audio reaches the host after a drop, and the tests below prove it through the real dictation hook and the real host controller.
Testing
Host (
runtime-mobile-dictation-controller.test.ts): a resent chunk is fed once and in spoken order, and finish feeds what's still held. A gap that never fills is given up on at the cap instead of stalling the transcript. Both tests fail on the pre-fix code.Phone (
mobile-dictation-reconnect.test.tsx): throughuseMobileDictation, a chunk lost to a drop is resent to a host with the flag, and never to a host without it. A recording cancelled during the outage is not resent. Each fails with its guard removed.Mobile RPC recording goldens for the dictation families replay clean. The two chunk goldens and the scenario were re-recorded for the new
seqparam.Focused dictation suites (mobile hooks, platform capture, session dictation; host controller, orcad speech services, speech worker lifetime) pass, as do the changed-file code-quality and React Doctor gates. Full lint, typecheck and test are left to CI.
Platforms: logic is platform-independent (phone transport + Node host); not hand-tested on a device.
I manually tested these changes locally
Automated tests added/updated, or explained why not below
AI Disclosure
Review
Agent skill upstream boundary
Notes
Ensure no issues in: Security, Cross-platform support (Linux, Windows, Mac), Remote SSH, Mobile, general backwards compatibility, performance
seqor a flood of tiny chunks can't make it loop, throw or grow past that.orca serveand orcad hosts use the same controller.For blessing (new behaviour: a host capability, and what the phone resends):
seq)./Users/kelvinamoaba/orca/workspaces/orca/dictation-dying-socket-audio(branchmobile-dictation-dying-socket).Checklist
N/Awith reasonpnpm lint,pnpm typecheck,pnpm test, andpnpm buildpass (or CI will cover; local preferred)