Translation: speak one language, type another - #25
Merged
Conversation
Cancelling an active dictation worked on every client already. On three of them nothing said so, which for a feature reachable only by a key or a gesture is the same as not having it: the desktops bound Escape and never printed it, and both phones cancelled capture by dragging off the talk button before letting go — undiscoverable, and with nowhere on a keyboard to be documented. What was left in practice was to let a recording you did not mean finish, pay for it, and delete what it typed. Phones get a control, shown only while there is something to abandon and named for which half of the dictation that is: Discard recording while the microphone is open, Cancel transcription once the request has gone. Two names rather than one because the two lose different things. On Android it is a cross beside the talk button — deliberately not on the utility row, which is always there and already has Backspace on it — and on iOS the keyboard's existing Cancel button simply stops hiding itself during recording. Both hosts route through one new DictationController.cancelActive(), because the state changes under the finger: a request starts the moment a recording ends, and a control that went inert at exactly that boundary would fail when it is most wanted. The desktops name the key instead of growing a button. Their overlay ignores the mouse on purpose — it hangs over whatever is being typed into, and a pill that swallowed clicks would take them from the application underneath — so the fix is that the pill says "Esc to cancel", in the recording phase and in the transcribing phase, where the wait is longer and the way out matters more. RecordingHint composes that row in the core of both desktops, with the same four cases asserted in Swift and C#: the key's *name* stays with the client, because macOS says Return where Windows says Enter, but the order and the join do not, or the row reads differently on the two platforms. swift test, dotnet test and ./gradlew test pass. The iOS UI tests pass with one pre-existing failure (testAPIKeySurvivesLeavingSettings), which reproduces on main; a new UI test covers discarding a recording. Android compiles and unit tests; untested on a device.
The space between Chinese and Latin was whatever the model felt like on the request. The same sentence came back spaced once and tight the next time, and sometimes with a stray space after a full-width full stop, which no convention allows anywhere. Every individual output was defensible; the set of them was not, and inconsistency is what was reported. The fix is split along the line between what can be decided locally and what cannot. Spacing is arithmetic, so it happens on the device. Typography runs in TranscriptionService after both audio guards and before the result leaves — the same choke point and the same argument as HallucinationGuard: dictation, file transcription, retry, redo and both CLIs all come through it, so history and the cursor cannot disagree, and a caller cannot forget it. One space at the boundary is the default, with "no space" and "leave it as the model wrote it" beside it; space next to a full-width mark is removed under both rules, because that one is not a preference. Three constraints keep it safe to run on every transcript. It may only add or remove horizontal space — all three suites assert the input and output are identical once whitespace is dropped. Newlines are not horizontal space, so it cannot join two lines. And Hangul is outside the CJK class while kana is inside it: Korean separates its own words, so "no space" would break the language, whereas excluding kana would space Web開発 and not Webかいはつ, which is the inconsistency the whole thing exists to remove. The 22-row table is repeated verbatim in Swift, C# and Kotlin, like the mode grammar. Script and punctuation are not arithmetic, so they are asked of the model. prompt/typography.md carries a chosen Chinese script — Simplified against Traditional was drifting inside single dictations — plus the request to separate clauses with full-width punctuation rather than spaces. prompt/sample.md carries the user's own formatting example, framed three times as an example and not as speech, the way screen context is. Replacing a space with a comma stays a request rather than a transform, because a comma the speaker did not say is a content change and this project does not make those locally. Both blocks are absent from a default request, and that is load-bearing rather than tidy: every measured number in docs/PROMPT.md describes the default request, and a clause added to it unconditionally would invalidate the whole table at once. The equality is a test, not a claim. Measured as a control rather than a before/after, since nothing sent a new block: 48 runs, 38 matched, 7 improved, 2 regressed, against the 2026-08-19 baseline's 37/11/2 — inside the per-pass range the runner prints for itself. The two regressions are the standing benefit-novel-repo failures. The measurement harness turns typography off explicitly, which is the second deliberate divergence in ARCHITECTURE.md: a suite scoring against ground truth transcribed from the backend's output should not be scoring this app's own transform. swift test (577), dotnet test (588) and ./gradlew test pass; iOS builds and its simulator tests pass. Android and Windows compile and unit-test; untested on a device.
7 tasks
CI's `dotnet format whitespace --verify-no-changes` rejected an attribute left on the same line as a property whose initialiser wraps. Caught by the Windows source-validation job, which is the one part of the matrix a Mac cannot run.
A target language in Settings, off by default, on all four clients and from `dnt transcribe --mode translate:English`. This is the one setting in the product that makes the main control deliver something other than what was said, so it is worth being exact about what it does not change. The verbatim transcript is still produced first, stored first, and recoverable — ⌘⌥Z, Ctrl+Alt+Z, the History row on both phones. That is the actual promise; "the main key stays verbatim" was how it had been kept so far, not the thing being kept. A translation you cannot expand back into the words that produced it is the failure this project was built against. One you can is a convenience sitting beside them. Three decisions worth the review time: It replaces the rewrite stage rather than joining it. One request doing two jobs is the combination already measured at twice the substitution rate, and "formal French" is a feature request rather than a fix for what was asked for. Every client says so where the rewrite control is, through a new RewriteAvailability.translating case, rather than leaving a picker that quietly does nothing — the same reason that type exists at all. It folds into the request that transcribes, wherever the backend can answer the wider schema. Same argument as the rewrite: that request is the only one holding the audio, and a translator working from text alone applies world knowledge and "corrects" a version number it believes is stale. Recognisers, split recordings and the live pipeline keep the second pass. StyledRequest replaces the bare style-clause parameter for this: a rewrite and a translation are different jobs asked for through the same field, and two optional parameters would have made "both at once" a state somebody has to remember not to construct. The language is free text with a shape check, exactly like the Model field and for the same reason. "Traditional Chinese", "Brazilian Portuguese" and "plain English" are all things a model can do and none is a row in an enum anybody would have thought to add. TranslationTarget checks shape, never existence; the suggestions are a shortcut and the suites assert they are not a whitelist. prompt/translate.md is its own part rather than a rewrite style, because the rewrite block's first rule is *keep the speaker's language* — the same reasoning that keeps the summary separate, where the rule at stake is "never remove a fact". Measured as a control, since the transcription request is untouched: 48 runs, 36 matched, 9 improved, 3 regressed. The regressed count moved from 2 and the honest reading is that it did not move — the runner printed `0–2 per pass` beside it and named real-version-number, the third regression, as one of two cases giving different answers across passes in that same run. swift test (582), dotnet test (607) and ./gradlew test pass; the mode grammar table gains eight translate rows in all three suites. iOS builds and its simulator tests pass, with the same pre-existing testAPIKeySurvivesLeavingSettings failure that reproduces on main. Android and Windows compile and unit-test; untested on a device.
`prompt show --section system` printed the bare contract while the app appends two more paragraphs to it, which makes the command useless for the one job it has — telling somebody what is being sent. It now reads the same two settings the app does, and gains a `translate` section beside `rewrite` and `summary`. Verified end to end against a real code-switched recording: with the spacing setting on `tight`, `一个retrieval pipeline，` and `这个index已经` come back tightened while the space inside "retrieval pipeline" is left alone.
Found reading the branch back. The Windows tray built its "what is happening" label as TranscriptMode.Rewrite(SecondaryStyle).ProgressLabel in two places, which is a second answer to a question the controller had already answered — so a translation showed "Loosening…" in the overlay and in the menu while a translation was in flight. AppSettings.SecondStageFor is now the one definition and both callers use it. The Kotlin PromptPart carried two consecutive KDoc blocks on isClause, because the new note was inserted ahead of the old one instead of into it.
bojieli
force-pushed
the
feat/translation
branch
from
August 30, 2026 11:00
0270fb3 to
406d9c3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes, and why
Requested: dictate in one language, get another at the cursor. A target language in Settings, off
by default, on all four clients and from
dnt transcribe --mode translate:English.This is the one setting in the product that makes the main control deliver something other than
what was said, so it is worth being exact about what it does not change. The verbatim
transcript is still produced first, still stored first, and still recoverable —
⌘⌥Z,Ctrl+Alt+Z, the History row on both phones. That is the actual promise; "the main key staysverbatim" was how it had been kept so far, not the thing being kept.
Three decisions worth the review time:
It replaces the rewrite stage rather than joining it. One request doing two jobs is the
combination already measured at twice the substitution rate, and "formal French" is a feature
request rather than a fix for what was asked for. Every client says so where the rewrite control
is, through a new
RewriteAvailability.translatingcase — that type exists precisely so a controlis never greyed out without a sentence.
It folds into the request that transcribes, wherever the backend can answer the wider schema.
Same argument as the rewrite: that request is the only one holding the audio, and a translator
working from text alone applies world knowledge and "corrects" a version number it believes is
stale. Recognisers, split recordings and the live pipeline keep the second pass.
StyledRequestreplaces the bare style-clause parameter for this. A rewrite and a translation aredifferent jobs asked for through the same
styledfield, and two optional parameters would havemade "both at once" a state somebody has to remember not to construct.
The language is free text with a shape check, exactly like the Model field and for the same
reason: "Traditional Chinese", "Brazilian Portuguese" and "plain English" are all things a model
can do and none is a row in an enum anybody would have thought to add.
TranslationTargetchecksshape and never existence; the suggestion list is a shortcut, and the suites assert it is not a
whitelist.
prompt/translate.mdis its own part rather than a rewrite style, because the rewrite block'sfirst rule is keep the speaker's language — the same reasoning that keeps the summary separate,
where the rule at stake is "never remove a fact".
Numbers, if this touches transcription quality
The new part is a second-stage instruction. It is never appended to the transcription
instruction, so the request carrying the audio is byte-identical whether or not a language is set.
The suite measures that request, so this ran as a control:
swift run dnt-eval suite eval/nearmiss --repeat-count 3, gemini-3.5-flash, 48 runs.The regressed count moved from 2 to 3 and the honest reading is that it did not move. The
runner prints its own per-pass range beside the figure —
REGRESSED 3 (0–2 per pass)— and namedreal-version-numberandbenefit-novel-repoas giving different answers across passes in thatsame run.
real-version-numberis the third regression and is one of the two. Nothing in therequest changed, so there is nothing for a change to have caused; what this records is the size of
the suite's noise.
benefit-novel-reporemains the standing failure it has been for weeks.Platforms
The mode-grammar parity table gains eight
translaterows in all three suites, including the twothat must be rejected (
translateandtranslate:— every other stage has an obvious default and"into what?" has none). The setting crosses devices in the transfer profile. On both phones the
mode chip says Translate and stops being a toggle while a language is set, because clearing it
belongs to the settings screen and a chip reading "Dictate" over a dictation coming back in another
language would be a lie.
swift test,dotnet testand./gradlew testpass. iOS builds and its simulator tests pass,with the same pre-existing
testAPIKeySurvivesLeavingSettingsfailure noted in #23.Updated after review: the series has since been run through
./gradlew connectedDebugAndroidTeston an API-35 emulator, so Android is no longer untested on adevice. A review pass over this branch also found the tray building its "what is happening" label
from
TranscriptMode.Rewrite(SecondaryStyle)in two places — a second answer to a question thecontroller had already answered, so a translation showed "Loosening…" while a translation was in
flight.
AppSettings.SecondStageForis now the one definition. Thedotnet formatfailure theWindows job reported here is fixed on #24, which this branch is rebased onto.
Checks
swift test,dotnet testand./gradlew testpass