Skip to content

[Bug]: An agent's click in a browser page commits the user's unfinished IME text into the terminal #28700

Closed

Description

@Fanzzzd

Operating system

macOS

Orca version

upstream main at 04e8cd9 (includes #28452, #28453, #28454)

Details

Summary. You are typing Chinese with pinyin in a terminal. You have typed zhong and not picked a candidate yet. An agent clicks something in an Orca browser page. Orca commits zhong into the terminal as plain letters. The candidate you then pick lands after it. The terminal prompt ends up with zhong中 instead of 中. In an agent prompt, that is a stray word in your message.

Steps (hidden-window e2e, ran twice, same result both times):

  1. Open a browser tab on a local page, then put the terminal back in front and focus it.
  2. Start a composition in the terminal: CDP Input.imeSetComposition with text zhong.
  3. The agent runs browser.click on a button in that page.
  4. Pick the candidate: CDP Input.insertText with 中.

Events on the terminal's .xterm-helper-textarea:

compositionstart, compositionupdate:zhong, input:zhong, compositionend:zhong, blur, focus, input:中

The prompt line reads % zhong中. Expected % 中.

Cause. docs/reference/agent-browser-focus-isolation.md says an agent press moves focus into the <webview>, and the pane gives focus back when the press ends. That blur ends the composition, and Chromium commits the unfinished text. The doc lists "an in-progress IME composition is lost" as a known gap. In a terminal the text is not lost: it is sent to the shell or the agent.

Comparison. The same probe on a page rendered offscreen (#24047) leaves the composition alone. Events are compositionupdate:zhong, then compositionupdate:中, then compositionend:中, and the prompt reads % 中.

Possible fix within the current design. The pane already records the user's focused element before an agent press. It could also report whether a composition is open: a compositionstart with no compositionend yet. Main could then hold the agent's press until the composition ends, with a timeout. I can send a PR if that direction works for you.

Limits of this test. The composition is driven over CDP, which enters Chromium at the same place as a real input method. I did not drive a real macOS input source, because switching it changes the input source for the whole machine.

Probe spec
/**
 * Temporary probe: does an agent click on a browser page break the IME composition the user is
 * in the middle of in the terminal?
 */
import { createServer, type Server } from 'node:http'
import type { Page } from '@stablyai/playwright-test'
import { test, expect } from './helpers/orca-app'
import { getTerminalContent } from './helpers/terminal-pane-identity'
import { ensureTerminalVisible, getActiveWorktreeId, waitForActiveWorktree } from './helpers/store'

const HTML = `<!doctype html><title>IME probe</title>
<button id="target" style="position:absolute;left:40px;top:80px;width:200px;height:60px">target</button>
<script>window.__clicks=0;document.getElementById('target').onclick=()=>{window.__clicks+=1}</script>`

async function call(page: Page, method: string, params: Record<string, unknown>): Promise<unknown> {
  const response = await page.evaluate(
    ({ m, p }) => window.api.runtime.call({ method: m, params: p }),
    { m: method, p: params }
  )
  expect(Reflect.get(Object(response), 'ok'), JSON.stringify(response)).toBe(true)
  return Reflect.get(Object(response), 'result')
}
const evalIn = async (page: Page, pageId: string, expression: string) =>
  Reflect.get(Object(await call(page, 'browser.eval', { page: pageId, expression })), 'result')

for (const offscreen of [false]) {
  test(`IME composition across an agent click (offscreen ${offscreen})`, async ({ orcaPage }) => {
    test.setTimeout(120_000)
    const server: Server = createServer((_q, r) => {
      r.writeHead(200, { 'Content-Type': 'text/html; charset=utf-8' })
      r.end(HTML)
    })
    await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve))
    const address = server.address()
    const url = `http://127.0.0.1:${typeof address === 'object' && address ? address.port : 0}/`
    try {
      await waitForActiveWorktree(orcaPage)
      await ensureTerminalVisible(orcaPage)
      await orcaPage.evaluate(
        (value) =>
          window.__store?.getState().updateSettings({ experimentalOffscreenBrowserPages: value }),
        offscreen
      )
      const worktreeId = await getActiveWorktreeId(orcaPage)
      const pageId = await orcaPage.evaluate(
        ({ id, target }) =>
          window.__store?.getState().createBrowserTab(id, target, { title: 'IME probe', activate: true })
            ?.activePageId ?? null,
        { id: worktreeId!, target: url }
      )
      await expect
        .poll(() => evalIn(orcaPage, pageId!, `document.readyState+':'+innerWidth+':'+window.__clicks`), {
          timeout: 20_000
        })
        .toMatch(/^complete:[1-9]\d*:0$/)
      await ensureTerminalVisible(orcaPage)

      await orcaPage.evaluate(() => {
        const box = document.querySelector<HTMLTextAreaElement>('.xterm-helper-textarea')!
        box.focus()
        const log: string[] = []
        const push = (entry: string) => {
          log.push(entry)
          document.documentElement.dataset.imeProbe = JSON.stringify(log)
        }
        document.documentElement.dataset.imeProbe = '[]'
        for (const type of ['compositionstart', 'compositionupdate', 'compositionend']) {
          box.addEventListener(type, (event) => push(`${type}:${(event as CompositionEvent).data}`))
        }
        box.addEventListener('input', (event) => push(`input:${(event as InputEvent).data}`))
        box.addEventListener('blur', () => push('blur'))
        box.addEventListener('focus', () => push('focus'))
      })
      const cdp = await orcaPage.context().newCDPSession(orcaPage)
      // The user is mid-composition: pinyin typed, candidate not chosen yet.
      await cdp.send('Input.imeSetComposition', { text: 'zhong', selectionStart: 5, selectionEnd: 5 })
      await orcaPage.waitForTimeout(200)

      await call(orcaPage, 'browser.click', { page: pageId, element: '#target' })
      await expect.poll(() => evalIn(orcaPage, pageId!, 'String(window.__clicks)')).toBe('1')
      await orcaPage.waitForTimeout(300)

      // The user picks the candidate.
      await cdp.send('Input.insertText', { text: '中' })
      await orcaPage.waitForTimeout(500)
      const result = await orcaPage.evaluate(() => ({
        log: JSON.parse(document.documentElement.dataset.imeProbe ?? '[]'),
        terminalFocused:
          document.activeElement?.classList.contains('xterm-helper-textarea') === true
      }))
      await orcaPage.waitForTimeout(1000)
      const screen = await getTerminalContent(orcaPage)
      const line = screen.split('\n').filter((row) => /zhong|中/.test(row)).join(' | ')
      console.log(`IME_TERMINAL offscreen=${offscreen} ${JSON.stringify(line)}`)
      console.log(`IME_PROBE offscreen=${offscreen} ${JSON.stringify(result)}`)
    } finally {
      await new Promise((resolve) => server.close(resolve))
    }
  })
}

Activity

  1. AmethystLiang commented on Oct 14, 2026

    @AmethystLiang
    Contributor

    Thanks for reporting. Looking into it

  2. added a commit that references this issue on Oct 18, 2026
    3b6c9b1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

os:macosIssues affecting macOS

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions