Skip to content

fix(terminal): keep a composition tail in the colours its cells had - #16024

Closed
ethznn wants to merge 3 commits into
stablyai:mainfrom
ethznn:fix/ime-preedit-tail-keeps-cell-attributes
Closed

ethznn wants to merge 3 commits into
stablyai:mainfrom
ethznn:fix/ime-preedit-tail-keeps-cell-attributes

Conversation

@ethznn

@ethznn ethznn commented Aug 23, 2026 •

Copy link
Copy Markdown
Contributor

ELI5

When you start typing a Korean syllable into an empty agent CLI prompt, the syllable is still being composed and has not reached the shell yet — so the CLI's grey hint text is still sitting on that row. Orca draws its own copy of that row next to the composing letter so the letter looks like it is pushing the text along, but it redraws the copy in the ordinary text colour. The grey hint stops looking grey and starts looking like something you already typed. It goes back to normal the instant the syllable lands. This makes Orca's copy keep the colours the row actually had.

What Changed

CompositionHelper built the rendered row tail with translateToString, which returns characters and drops every cell attribute. It now reads the cells and emits the tail as runs that share a rendered style, each carrying what the grid resolved for it: palette, RGB and default foregrounds, backgrounds, dim, bold, italic, inverse, invisible, underline, strikethrough, overline.

A run that needs no override stays a bare text node and keeps inheriting the view's colour, so a default tail emits exactly the DOM it did before. The end column still comes from getTrimmedLength(), so the tail covers the columns it always did.

The refresh that keeps an open tail current keyed off the rendered text, which cannot see a repaint that recolours a row without changing its characters. It now keys off text and styling together.

Files: the xterm source patch, its regenerated bundle patch, the pnpm-lock.yaml hash bump, and one new test file.

Why

The tail exists so a mid-line composition reads as inserted text pushing the row right rather than an opaque box hiding the character under the cursor (#12545). The view paints one themed colour over everything it covers:

this._compositionView.style.color = this._optionsService.rawOptions.theme?.foreground ?? '#FFF';

That is correct for the preedit and for default cells. Any cell carrying its own attributes came back in the default foreground — a dim placeholder here, but equally a coloured prompt or a highlighted segment.

Reading the cells is the only way to render them faithfully; there is no smaller correct version. The DOM renderer's own xterm-fg-* and xterm-dim classes cannot be reused, because they are scoped to the rows container and only generated while that renderer is the active one.

Linked Issue

Fixes #16025

Visual Proof

Before, both on v1.4.188 with the input empty, so an open composition is the only difference. Codex is the CLI; its empty-input placeholder is Ask Codex to do anything.

At rest — the placeholder is dim, which is how it is meant to read:

At rest

Composing ㅇ — the placeholder is drawn as ordinary text:

Composing

Measured rather than eyeballed. Glyph pixels in the to do anything span — no block cursor and no jamo inside it — on the prompt row only, against the same background (luminance 52):

glyph pixels mean p90 max
at rest 388 116 143 221
composing 390 186 254 254

Same string in the same font, and the glyph counts land within two of each other, so this is brightness alone: about 60% brighter for as long as the composition stays open.

After, this branch built and run with the same CLI, composing the same ㅇ:

After the fix, composing

And the same branch at rest, as a control that the ordinary case did not move:

After the fix, at rest

Measured the same way — same span, same prompt-row restriction so the status line below cannot leak in:

at rest composing
v1.4.188 116 186
this branch 118 122

Composing used to land 70 above its own at-rest baseline; it now lands 4 above, and p90 falls from 254 to 154. The at-rest column moving 116 → 118 is the control: a default row renders as it always did.

Worth saying plainly — before and after look nearly identical at a glance, because the glyphs are in the same places and only their brightness differs. That is the point. Nothing shifts, nothing disappears; the tail just stops being repainted.

For contrast, iTerm2 with the same CLI keeps the placeholder dim but has no rendered tail at all, so its preedit box hides the cells under it — Ask is gone and the row reads , Codex to do anything. That is the occlusion #12545 was about, and it is why reverting to "do not draw a tail" is not the fix.

Testing

Regenerated per docs/reference/xterm-patch-regeneration.md: source hunks edited in the generator's checkout, --write, pnpm install, then --check reports in sync (7406065 bytes). Source patch, bundle patch and the lockfile hash are in one commit, as that document requires.

Command Result
pnpm test 6084 files, 57270 tests passed, 40 files / 254 tests skipped
pnpm lint 0 warnings, 0 errors
pnpm typecheck clean
pnpm build clean
tsgo --noEmit -p src/browser/tsconfig.json in the patched upstream tree clean

One file failed the first full run — tests/e2e/cross-version-wire/cross-version-terminal-wire.unit.test.ts, which needs release tags my --depth 1 clone did not have. Its own failure message names that cause. After git fetch --tags it passes 5/5.

terminal-ime-xterm-preedit-tail-attributes.test.ts pins seven cases: a dim tail stays dim, a palette colour survives, a default tail emits no run of its own, the tail splits where styling changes, bold and italic carry through, a recolour with identical characters still refreshes, and trailing blanks are still dropped. The third and sixth are the ones that would catch a regression in this change rather than restate it — one guards against styling cells that need none, the other against the text-only comparison the old refresh used.

The existing midline-preedit-tail and trailing-preedit-occlusion suites are unchanged and pass: the view keeps its own colour and the overrides sit on child runs, so what they assert still holds.

  • I manually tested these changes locally
  • Automated tests added/updated, or explained why not below

Manual pass: reproduced on the shipped v1.4.188, then built and ran this branch and repeated it with a live 2-Set Korean IME, on macOS. Both states above are from those two runs, and the patched bundle is what the run loaded — setThemeService is present in node_modules/@xterm/xterm/lib/xterm.js and .mjs.

AI Disclosure

Claude Opus 5 via Claude Code, used throughout: locating the cause in the xterm patch, writing the change and its tests, and running the gates above. Every measurement quoted here — the luminance figures, the --check byte count, the suite results — was produced by running the command, not estimated. The reproduction and all four screenshots are mine, on my own machine.

Review

Code review summary, per CONTRIBUTING:

  • Cross-platform. No platform branch is touched. The tail path runs wherever a composition opens, so Windows and Linux IMEs get the same fix. getTrimmedLength() and the cell accessors are platform-neutral.
  • Remote / SSH. Rendering only; nothing crosses the wire and no PTY bytes change. A remote session renders from the same local buffer.
  • Agents and integrations. Any TUI that styles the row benefits; the CLI in the repro is Codex, and Claude Code and Cursor Agent place placeholders the same way. Nothing keys off which agent is running.
  • Performance. One extra pass over the row tail per composition render, bounded by cols. The previous code already walked the same span inside translateToString, and the refresh check now walks once where it used to walk once and compare, so the work is comparable. The CellData instance is allocated per helper and reused per cell, so a render allocates nothing per cell. No timer, listener or layout read is added.
  • UI quality. Default cells emit byte-identical DOM, so the common tail cannot regress. Dim is applied as half opacity on the resolved foreground, matching how the renderer fades a dim cell.
  • Security. No new input sink, no network, no filesystem, no eval. Cell text goes in through textContent, never innerHTML; the only attribute written is style, built from a closed set of enum branches with numeric colour channels, so nothing user-controlled reaches a CSS value.
  • Backwards compatibility. The theme arrives through a setter and the helper falls back to today's flat colour until it is handed over, so an older or partial construction path degrades to current behaviour rather than throwing.

Known limitation, called out deliberately: minimum contrast is not applied to a tail run. The renderer adjusts a cell against its own background; doing that here would need the same cache and still resolve against a different background. Leaving it out keeps every cell closer to the grid than one flat colour was. Glad to add it if you would rather have the exact renderer path.

Agent skill upstream boundary

  • Not applicable, or this change follows docs/reference/agent-skill-sharing-upstream-boundary.md and copies or mechanically translates no upstream skill-installer source, tests, fixtures, registry entries, path tables, comments, or documentation.

Notes

The theme is handed over by a setter rather than injected. Adding @IThemeService to the constructor needs a hunk in upstream's CompositionHelper.test.ts, which .npmignore strips from the published tarball — docs/reference/xterm-patch-regeneration.md says such a hunk fails the generator rather than being silently dropped. CoreBrowserTerminal hands the service over right after it creates the helper, which is where the existing patch already reaches in, and IThemeService is registered well before that point.

Checklist

  • This PR is small and focused
  • I explained what changed and why (including ELI5)
  • Before/after screenshots or videos attached for UI changes, or N/A with reason
  • Self-reviewed for correctness, security, and performance
  • Cross-platform, SSH/remote, and path/shortcut impact considered (or N/A)
  • pnpm lint, pnpm typecheck, pnpm test, and pnpm build pass (or CI will cover; local preferred)

A mid-line composition renders the rest of the row inside `.composition-view`
so the preedit reads as inserted text pushing the tail right (stablyai#12545). The tail
came from `translateToString`, which returns characters and drops every cell
attribute, and the view paints one themed colour over everything it covers — so
any cell not drawn in the default foreground came back in it.

The visible case is composing the first Hangul syllable into an empty agent CLI
prompt. The syllable is still in the preedit, so nothing reached the pty and the
CLI's dim placeholder is still on the row; the tail redraws it at full
brightness and it reads as text the user had typed. Typing on clears it, and
Enter submits only the syllable, which places the defect in the overlay rather
than in the buffer. Latin input never shows it, because a committed character
empties the placeholder before any tail is rendered.

Read the cells instead, and emit the tail as runs that share a rendered style,
each carrying the colour and flags the grid resolved for it: palette, RGB and
default foregrounds, backgrounds, dim, bold, italic, inverse, invisible,
underline, strikethrough, overline. A run that needs no override stays a bare
text node and keeps inheriting the view's colour, so the default tail emits
exactly the DOM it did before. The end column still comes from
`getTrimmedLength()`, so the tail covers the columns it always did.

The tail refresh keyed off the rendered text, which cannot see a repaint that
recolours a row without changing it, so it now keys off text and styling
together.

The theme arrives through a setter rather than the constructor: widening the
constructor needs a hunk in upstream's `CompositionHelper.test.ts`, which
`.npmignore` strips from the published tarball and the generator therefore
cannot carry. Before it is handed over, a tail renders as it does today. The DOM
renderer's own `xterm-fg-*` and `xterm-dim` classes are not reusable here —
they are scoped to the rows container and only generated while that renderer is
the active one.

Minimum contrast is not applied to a tail run. The renderer adjusts a cell
against its own background; doing that here would need the same cache and still
resolve against a different background, and leaving it out keeps every cell
closer to the grid than one flat colour was.
@coderabbitai

coderabbitai Bot commented Aug 23, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 97e7bad4-9c1a-4dc3-aaa0-43e3afa5464c

📥 Commits

Reviewing files that changed from the base of the PR and between 5fe584a and abe724a.

📒 Files selected for processing (1)
  • src/renderer/src/components/terminal-pane/terminal-ime-xterm-preedit-tail-attributes.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

The patch adds transaction-based IME composition handling across composition, keypress, input, blur, and disposal events. It prevents duplicate commits and handles cancellation and stale transactions. Composition overlays render styled terminal tail cells and update their positioning after repaints. CoreBrowserTerminal injects theme state and manages CompositionHelper lifecycle. SortedList deletion retries after pending cleanup changes sort keys. A test suite covers composition-tail styling and repaint behavior.

Merge Risk: ⚪ Minimal · up to abe72

The change preserves each cell’s existing colors while rendering an active composition tail, without changing terminal input or network behavior. No actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The patch also changes composition transaction handling, lifecycle cleanup, public APIs, and SortedList.delete behavior beyond issue #16025. Remove or separately justify the unrelated composition lifecycle, transaction, API, and SortedList.delete changes.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the terminal composition-tail color fix.
Description check ✅ Passed The description completes the required sections and documents the change, rationale, visual proof, testing, and checklist.
Linked Issues check ✅ Passed The changes directly address issue #16025 by preserving cell styling in the composition tail and refreshing on style-only changes.
Docstring Coverage ✅ Passed Docstring check was indeterminate for this PR — some files could not be analyzed in time. Not blocking.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
config/patches/xterm-src/@xterm__xterm@6.1.0-beta.287.src.patch (1)

1157-1162: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Guard the tail refresh on overlay state, not on preedit text.

The refresh runs only when this._compositionViewData is truthy. An IME can send compositionupdate with a single-character preedit and later a repaint, which is covered. But a composition whose current preedit string is empty while the view stays active keeps _compositionViewData === '', so a row repaint no longer re-renders the tail and the overlay keeps stale cells until the next composition event. Track a separate "tail rendered" flag or compare against _compositionRemainderKey !== undefined instead.

♻️ Proposed guard
-    if (
-      this._compositionViewData &&
-      this._getRowRemainder().key !== (this._compositionRemainderKey ?? '')
-    ) {
+    if (
+      this._compositionViewData !== undefined &&
+      this._getRowRemainder().key !== (this._compositionRemainderKey ?? '')
+    ) {
       this._renderCompositionView(this._compositionViewData);
     }

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b3dacf67-3952-40dd-ba0d-69760c260b15

📥 Commits

Reviewing files that changed from the base of the PR and between 063b804 and b043006.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (3)
  • config/patches/@xterm__xterm@6.1.0-beta.287.patch
  • config/patches/xterm-src/@xterm__xterm@6.1.0-beta.287.src.patch
  • src/renderer/src/components/terminal-pane/terminal-ime-xterm-preedit-tail-attributes.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

The trailing-blank case wrote the same data as the default-tail case above it
and asserted a subset of it, under a comment about erasing to end of row that
no escape sequence backed. It could not fail for the reason it named.

Replaced with a row that carries a run of spaces inside the text. That pins the
two things the tail's end column and its `white-space: pre` are for, and both
arms bite: reverting the trimmed end column to the full row length fails it, as
does letting the nowrap view collapse the inner run.
@ethznn

ethznn commented Aug 23, 2026

Copy link
Copy Markdown
Contributor Author

I tried this and then backed it out, because the state it protects against is not reachable.

An empty preedit does not leave the overlay up. compositionupdate hides it in the same call that renders it:

this._renderCompositionView(ev.data ?? '');
// Empty data is the IME saying the preedit is gone, so it hides the view again rather than
// leaving an orphaned overlay.
this._compositionView.classList.toggle('active', Boolean(ev.data));

updateCompositionElements returns early unless the view carries active, so with an empty preedit there is no refresh to run and nothing on screen to go stale. _renderCompositionView has exactly two call sites — that one and the refresh itself — and active is only ever set there and in compositionstart, which does not render. So before the first update there is no rendered tail either, and skipping is correct rather than incidental.

That makes the two guards behaviourally identical in every reachable state. I wrote the change and a test for the empty-preedit repaint; the test failed against the fix for the same reason it failed without it, which is what sent me looking at the toggle.

On the suggested diff specifically: _compositionViewData is set to '' by _resetCompositionView, not to undefined, so !== undefined stays true after a reset. Nothing breaks today because of the same active check, but it would not be narrowing what the truthiness check narrows.

Keeping the original guard. It is also worth noting the truthiness check predates this PR — only the comparison inside it is mine — so switching it would have put a second 7 MB regenerated bundle into a PR about tail colours, for no behaviour change. If you can name a path that reaches active with a rendered tail and empty data, I will take it; I could not find one.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 785ed4c8-8f4b-4a8d-8852-101e70aeff8d

📥 Commits

Reviewing files that changed from the base of the PR and between b043006 and 5fe584a.

📒 Files selected for processing (1)
  • src/renderer/src/components/terminal-pane/terminal-ime-xterm-preedit-tail-attributes.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review.

The spaces case read `textContent`, which is unaffected by CSS whitespace
handling, so dropping `white-space: pre` from the rendered tail left the whole
file green while every glyph after the run moved a cell left.

Assert the style alongside the text. Mutating the loaded bundle's
`whiteSpace = 'pre'` to `'normal'` now takes the case down; before this it took
nothing down.
@lee-eojin

Copy link
Copy Markdown

Thanks for the fix and the clear before/after proof. Preserving the dim styling is definitely an improvement.

Would it also be possible or should it be tracked separately to hide the placeholder as soon as Korean IME composition starts, matching the behavior after the first Latin character? That would address the original UX issue I noticed most directly.

@ethznn

ethznn commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

Separate, I think — the placeholder belongs to Codex. A composing syllable never reaches the pty, so its buffer is still empty and it keeps drawing the hint; after a Latin character it clears the hint itself.

Orca draws the copy you see, but it can't tell a placeholder that should vanish from real text that must stay shifted right. And dropping the tail while composing blanks whatever sits under the overlay — the occlusion #12545 was filed for. Worth raising upstream on Codex if you want it tracked.

@ethznn

ethznn commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #17631.

#17170 moved the pinned xterm to 6.1.0-beta.303 and reworked _renderCompositionView, so the patch file this PR edits no longer exists and there is nothing to rebase onto. The replacement is cut against beta.303, and the defect is unchanged there — _getRowRemainderText still returns plain text and the view still paints one colour over it.

It also came out smaller: CompositionHelper already takes @IThemeService on that version, so the setter this PR needed is gone.

@lee-eojin — your question carried over to the new one, and thanks for the read here.

@ethznn ethznn closed this Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: A composing syllable redraws the row tail in the default foreground, so a dim placeholder reads as entered text

2 participants