Skip to content

Fix mobile live input keyboard and Korean IME - #7273

Merged
Jinwoo-H merged 8 commits into
mainfrom
Jinwoo-H/fix-mobile-live-keyboard-ime
Jul 3, 2026
Merged

Jinwoo-H merged 8 commits into
mainfrom
Jinwoo-H/fix-mobile-live-keyboard-ime

Conversation

@Jinwoo-H

@Jinwoo-H Jinwoo-H commented Jul 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Fixes Korean Samsung Keyboard live input by using the Hangul mirror model from Fix mobile terminal Korean typing left invisible by the IME deferral model #7181: stream stable text immediately, hold the trailing composing Hangul code point, flush on settle/submit/accessory keys, and serialize live sends so deltas stay ordered.
  • Keeps pending Korean composition through transient active-tab lag instead of dropping it while the mobile tab snapshot catches up.
  • Makes the live input status row a visible, accessible keyboard target: the "Live input" row now looks tappable, says "Tap to show keyboard", has pressed/disabled states, and focuses the hidden capture TextInput.
  • Handles the real-device reopen case where Android leaves the hidden live-input TextInput focused after the IME is dismissed: tapping the live input target now blurs and schedules a fresh focus when the keyboard is closed.

Closes #7183.
Supersedes #7181 by carrying those commits forward and adding the keyboard affordance/refocus fix.

Testing

  • pnpm --dir mobile test — 170 files passed, 1231 passed / 2 skipped
  • pnpm --dir mobile lint
  • pnpm --dir mobile run format:check
  • pnpm --dir mobile exec tsc --noEmit
  • NODE_ENV=development ./gradlew :app:assembleDebug from mobile/android — BUILD SUCCESSFUL
  • pnpm --dir mobile run android — BUILD SUCCESSFUL, installed and opened on SM_S921W / R3CX105QXRH

Device QA

  • Installed/opened the debug build on the attached Android device after the refocus follow-up.
  • Manual verification requested from the device owner for: close keyboard → tap "Tap to show keyboard" → keyboard reopens.

realitsyourman and others added 7 commits July 3, 2026 18:29
탭 상태 지연으로 활성 탭 타입이 일시적으로 null이 될 때 runMirrorStep의 stale-handle 가드가 조합 중 음절을 버리지 않도록 pending-clear 효과와 동일한 null 허용 패턴 적용. 테스트 하네스가 ref와 prop을 동일 소스에서 파생하도록 결합해 실제 경로의 lag 프레임을 검증. queueTerminalLiveMirrorSend의 previousSend await를 catch로 보호.
… the mirror model

The mirror model migrated all live-input sends to queueTerminalLiveMirrorSend,
leaving queueTerminalLivePendingFlush referenced only by its own tests. Remove
the dead function and its three tests.

Co-authored-by: Orca <help@stably.ai>
Co-authored-by: Orca <help@stably.ai>
@coderabbitai

coderabbitai Bot commented Jul 3, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR reworks mobile terminal live input handling from a single pending-text/timer model to a held/sent mirror model. It adds a Hangul mirror utility module for composing erase/append payloads, updates the mirror send queue, special-key and accessory decision logic, and the pending-input-flush, accessory-commit, and live-input-commit hooks accordingly. It also updates the live input focus button styling/accessibility, TextInput props, and status text, along with corresponding test suites across affected modules.

Sequence Diagram(s)

Diagrams included per-layer in the hidden review stack artifact.

Related Issues: Unable to determine from provided information.

Related PRs: Unable to determine from provided information.

Suggested labels: mobile, terminal, refactor

Suggested reviewers: Unable to determine from provided information.

Poem:
A rabbit tapped the terminal glass,
Held each Hangul syllable, let none pass,
DEL by DEL, the mirror stayed true,
Sent text and held text, a tidy queue,
Hop, commit, and send — the old logic's gone at last.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description covers summary and testing, but it omits the required Screenshots, AI Review Report, Security Audit, and Notes sections. Add the missing template sections, including screenshots or "No visual change", an AI review report, a security audit, and notes.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes implement the Hangul mirror flow, preserve transient tab lag, and add the keyboard-affordance/refocus fixes requested by #7183.
Out of Scope Changes check ✅ Passed No clearly unrelated code changes stand out; the UI, refocus, and test updates all support the Korean IME/mobile live-input fix.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check ✅ Passed The title clearly reflects the main change: fixing mobile live input behavior for keyboard handling and Korean IME support.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
mobile/app/h/[hostId]/session/[worktreeId].tsx (1)

4982-4990: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Missing "why" comment for showSoftInputOnFocus on the hidden capture input.

This prop directly implements the mobile IME fix (showing the keyboard only via the visible focus affordance while the actual capture field stays hidden/absolute-positioned). As per coding guidelines, non-obvious design-driven behavior should carry a short explanatory comment, but none is present here.

📝 Suggested comment
                       placeholder=""
+                      // Why: the capture input is visually hidden; showSoftInputOnFocus
+                      // ensures tapping the visible keyboard affordance still raises the IME.
                       showSoftInputOnFocus
As per coding guidelines, "When writing or modifying code driven by a design doc or non-obvious constraint, add a comment explaining why the code behaves the way it does."

Source: Coding guidelines

🧹 Nitpick comments (2)
mobile/src/terminal/terminal-live-hangul-mirror.ts (1)

34-39: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Minor: redundant undefined check.

holdLast already requires lastCodePoint !== undefined (line 36), so the repeated lastCodePoint !== undefined check at line 38 is dead logic. Same applies to the harmless ?? 0 fallback at line 37 since Array.from never yields an empty grapheme string. Purely cosmetic, not worth a dedicated change.

mobile/app/h/[hostId]/session/[worktreeId].tsx (1)

4949-4966: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Static accessibilityLabel masks dynamic dictation/attach status for screen readers.

The Pressable's accessibilityLabel is now a fixed string ("Show keyboard for live terminal input"), but it wraps MobileTerminalLiveInputStatus, whose text dynamically reflects "Listening", "Processing", "Uploading image to host", etc. Since an explicit accessibilityLabel overrides the accessible name derived from children, screen-reader users tapping this control while dictation/attachment is active will only hear the static label and lose the live status context that sighted users see.

Consider deriving the label from the current status (e.g. reuse the detail string) or exposing it via accessibilityValue/accessibilityHint so VoiceOver/TalkBack conveys state changes.

♿ Possible fix sketch
-                      accessibilityLabel="Show keyboard for live terminal input"
+                      accessibilityLabel={
+                        dictation.isRecording || dictation.isProcessing || dictation.isStarting || isAttaching
+                          ? 'Live terminal input status'
+                          : 'Show keyboard for live terminal input'
+                      }

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 0efda7df-3a97-42ff-b0cf-4c8403e76410

📥 Commits

Reviewing files that changed from the base of the PR and between 25896f2 and b69e9f7.

📒 Files selected for processing (15)
  • mobile/app/h/[hostId]/session/[worktreeId].tsx
  • mobile/app/h/[hostId]/session/mobile-session-command-input-styles.ts
  • mobile/src/session/MobileTerminalLiveInputStatus.tsx
  • mobile/src/terminal/terminal-live-hangul-mirror.test.ts
  • mobile/src/terminal/terminal-live-hangul-mirror.ts
  • mobile/src/terminal/terminal-live-input-affordance.test.ts
  • mobile/src/terminal/terminal-live-pending-flush-state.test.ts
  • mobile/src/terminal/terminal-live-pending-flush-state.ts
  • mobile/src/terminal/terminal-live-text-commit.test.ts
  • mobile/src/terminal/terminal-live-text-commit.ts
  • mobile/src/terminal/use-terminal-live-accessory-input-commit.test.ts
  • mobile/src/terminal/use-terminal-live-accessory-input-commit.ts
  • mobile/src/terminal/use-terminal-live-input-commit.test.ts
  • mobile/src/terminal/use-terminal-live-input-commit.ts
  • mobile/src/terminal/use-terminal-live-pending-input-flush.ts

Comment on lines +268 to +278
it('Given Backspace with field text When the key arrives Then edits locally without terminal bytes', async () => {
// Given
const { handlers, sent } = createTerminalLiveInputCommitHarness()
handlers.handleLiveInputChange('한')

// When
handlers.handleLiveInputKeyPress({ nativeEvent: { key: 'Backspace' } })

// Then
await vi.waitFor(() => expect(sent).toEqual([]))
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Simulate the native text change after Backspace.

This path relies on onChangeText to clear the held syllable/timer; otherwise the test leaves the held settle timer alive and only asserts before it can fire.

Proposed test adjustment
 handlers.handleLiveInputKeyPress({ nativeEvent: { key: 'Backspace' } })
+handlers.handleLiveInputChange('')
 
 // Then
 await vi.waitFor(() => expect(sent).toEqual([]))
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
it('Given Backspace with field text When the key arrives Then edits locally without terminal bytes', async () => {
// Given
const { handlers, sent } = createTerminalLiveInputCommitHarness()
handlers.handleLiveInputChange('한')
// When
handlers.handleLiveInputKeyPress({ nativeEvent: { key: 'Backspace' } })
// Then
await vi.waitFor(() => expect(sent).toEqual([]))
})
it('Given Backspace with field text When the key arrives Then edits locally without terminal bytes', async () => {
// Given
const { handlers, sent } = createTerminalLiveInputCommitHarness()
handlers.handleLiveInputChange('한')
// When
handlers.handleLiveInputKeyPress({ nativeEvent: { key: 'Backspace' } })
handlers.handleLiveInputChange('')
// Then
await vi.waitFor(() => expect(sent).toEqual([]))
})

Co-authored-by: Orca <help@stably.ai>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
mobile/src/terminal/terminal-live-input.ts (1)

231-249: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Android-only workaround applied unconditionally to all platforms.

The Why comment explains this blur+refocus dance is needed because "Android can keep a hidden TextInput focused after the IME is dismissed," but the branch at Line 239 executes on any platform where keyboardHeight <= 0 && input.isFocused?.() — including iOS. If that condition is ever true on iOS (e.g. keyboard dismissed by a hardware/external-keyboard event while the hidden capture input retains focus), this forces an extra blur → scheduled refocus cycle that iOS doesn't need, which is a behavior only intended for Android running unconditionally.

As per coding guidelines, Keep all platform-dependent behavior behind runtime checks.

🩹 Suggested platform gate
+import { Platform } from 'react-native'
+
 export function focusTerminalLiveInputTarget(
   input: TerminalLiveInputFocusTarget | null,
   { keyboardHeight, refocus }: FocusTerminalLiveInputTargetOptions
 ): void {
   if (!input) {
     return
   }

-  if (keyboardHeight <= 0 && input.isFocused?.()) {
+  if (Platform.OS === 'android' && keyboardHeight <= 0 && input.isFocused?.()) {
     // Why: Android can keep a hidden TextInput focused after the IME is dismissed;
     // focus() is then a no-op, so force a new focus session to reopen the keyboard.
     input.blur()
     refocus()
     return
   }

   input.focus()
 }

Please confirm whether this branch was intentionally left cross-platform (e.g. if iOS can also hit this hidden-focus state) — if so, please adjust the comment to reflect that; otherwise gate it to Android.

Source: Coding guidelines


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 76618a80-f966-464d-8f0b-098110b3f2dc

📥 Commits

Reviewing files that changed from the base of the PR and between b69e9f7 and cfb685b.

📒 Files selected for processing (4)
  • mobile/app/h/[hostId]/session/[worktreeId].tsx
  • mobile/src/terminal/terminal-live-input-affordance.test.ts
  • mobile/src/terminal/terminal-live-input.test.ts
  • mobile/src/terminal/terminal-live-input.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • mobile/src/terminal/terminal-live-input-affordance.test.ts

@Jinwoo-H
Jinwoo-H merged commit e8c2b79 into main Jul 3, 2026
2 checks passed
@Jinwoo-H
Jinwoo-H deleted the Jinwoo-H/fix-mobile-live-keyboard-ime branch July 3, 2026 23:48
brennanb2025 added a commit that referenced this pull request Jul 30, 2026
#6995 turned out to be already fixed on main by #7011 (4f0a141) and
#7273 (e8c2b79): a physical Samsung with Samsung Keyboard composes
Hangul correctly on a build that does not contain the omission change.
Shipping it anyway would surface the suggestion strip over the invisible
live-input buffer to fix nothing. What stays is the #4606 half: the
tri-state autocomplete preference, autocorrect defaulting on for the iOS
command bar, and the props-derived Android remount key.
enrick-dev pushed a commit to enrick-dev/orca that referenced this pull request Oct 15, 2026
* feat: 모바일 터미널 한글 미러 스텝 순수 모델 추가

* feat: 미러 델타 순서 보장용 send 체인 추가

* fix: 모바일 터미널 한글 입력을 미러 모델로 전환

* fix: 탭 상태 지연 중 한글 조합 상태 소실 방지

* fix: 미러 가드와 send 체인 리뷰 지적사항 반영

탭 상태 지연으로 활성 탭 타입이 일시적으로 null이 될 때 runMirrorStep의 stale-handle 가드가 조합 중 음절을 버리지 않도록 pending-clear 효과와 동일한 null 허용 패턴 적용. 테스트 하네스가 ref와 prop을 동일 소스에서 파생하도록 결합해 실제 경로의 lag 프레임을 검증. queueTerminalLiveMirrorSend의 previousSend await를 catch로 보호.

* refactor(mobile): drop dead queueTerminalLivePendingFlush orphaned by the mirror model

The mirror model migrated all live-input sends to queueTerminalLiveMirrorSend,
leaving queueTerminalLivePendingFlush referenced only by its own tests. Remove
the dead function and its three tests.

Co-authored-by: Orca <help@stably.ai>

* fix(mobile): expose live terminal keyboard target

Co-authored-by: Orca <help@stably.ai>

* fix(mobile): refocus live keyboard after dismissal

Co-authored-by: Orca <help@stably.ai>

---------

Co-authored-by: realitsyourman <wongil@demodev.io>
Co-authored-by: Neil <4138956+nwparker@users.noreply.github.com>
Co-authored-by: Orca <help@stably.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Mobile terminal Korean IME input is completely invisible after the #7011 deferral model

3 participants