Skip to content

feat(cua): attach the managed Cua Driver to every provider session - #16528

Draft
juliusmarminge wants to merge 1 commit into
t3code/cua/1-hostfrom
t3code/cua/2-providers
Draft

juliusmarminge wants to merge 1 commit into
t3code/cua/1-hostfrom
t3code/cua/2-providers

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

With the driver running (previous layer), every provider now gets it as a stdio MCP server named cua-driver:

Provider How
Claude SDK mcpServers; the query reopens when the set changes
Codex thread mcp_servers, unless the user's merged Codex config already defines cua-driver (read once per directory with config/read); theirs wins
Cursor SDK mcpServers
Grok, Antigravity, Devin, ACP registry ACP mcpServers, in both stdio and MCP-over-ACP lists
OpenCode 1.x mcp.add; the prompt mentions it only if the add succeeded
OpenCode 2 per-thread, per-directory registration with permission rules, like its t3-code server
Pi Pi has no MCP client, so T3's Pi extension bridges the stdio server and passes screenshots through as images

Sessions with the driver get a short <computer_use> block: keep input in the background, escalate to foreground only when a background action did not land, and leave the desktop as found.

#15006 skipped Codex, OpenCode 2, and Pi. #10899 only attached Codex. This covers all of them.

#15006 also ran a per-thread proxy so T3 could end Cua's session at every turn end. This stack drops it: Stop still ends the provider's turn, and Cua's own idle timeout hides the agent cursor.

Verification

  • Focused tests: Claude and Codex attachment, user-config precedence in Codex, the <computer_use> block, and the existing Claude, Codex, Cursor, ACP, Grok, OpenCode, OpenCode 2, Pi, and session-manager suites (687 tests).
  • Checked by hand: Pi's stdio bridge against the real 0.34 driver lists 58 tools, calls list_apps, and reconnects after the driver exits.

Built with Claude Opus 5.5 in T3 Code, through the Claude Code harness.

🤖 Generated with Claude Code

@juliusmarminge juliusmarminge added the macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews label Oct 6, 2026
@juliusmarminge
juliusmarminge added this pull request to stack #16533 October 6, 2026 17:44
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Oct 6, 2026
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire — 5.0 KiB — 6.8 KiB ✅
Codex Thread snapshot wire — 3.8 KiB — 4.9 KiB ✅
Codex Live turn WebSocket wire — 1.2 KiB — 2.0 KiB ✅
Codex Live turn WebSocket decoded — 20.9 KiB — 29.3 KiB ✅
Codex Live turn messages — 2 — 8 ✅
Claude Total thread wire — 5.0 KiB — 6.8 KiB ✅
Claude Thread snapshot wire — 3.8 KiB — 4.9 KiB ✅
Claude Live turn WebSocket wire — 1.2 KiB — 2.0 KiB ✅
Claude Live turn WebSocket decoded — 21.2 KiB — 29.3 KiB ✅
Claude Live turn messages — 2 — 8 ✅

Baseline: unavailable · PR result: 79285b2 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

The session manager records the environment's running Cua Driver on each
thread's MCP session, and every adapter attaches it as a stdio server named
`cua-driver`: Claude, Cursor, the ACP family (Grok, Antigravity, Devin, the
ACP registry), OpenCode 1.x, OpenCode 2 (per thread, like its T3 server),
and Codex, whose own `cua-driver` config still wins. Pi has no MCP client,
so T3's Pi extension bridges the stdio server and passes screenshots through
as images.

Sessions with the driver get a short <computer_use> block that keeps input
in the background and asks the agent to leave the desktop as it found it.

Ported from #10899 (Francesco Bonacci, Julius) and #15006 (Theo).

Co-Authored-By: Francesco Bonacci <f@trycua.com>
Co-Authored-By: Theo Browne <me@t3.gg>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@juliusmarminge
juliusmarminge force-pushed the t3code/cua/2-providers branch from 39c4a19 to 79285b2 Compare October 6, 2026 20:54
kvnloo added a commit to kvnloo/t3code that referenced this pull request Oct 12, 2026
The async child_process import permits concurrent Pi tool requests to
spawn multiple clients after the driver exits. Share the pending start
and initialization, reset handshake status on exit/close, and keep
reconnect explicitly session-scoped. Fork-only CHECK candidate for
Julius Marminge's existing pingdotgg#16528; source finding PER-1778.
GPT-6 / ChatGPT under Kevin's direction; no upstream PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews size:L 100-499 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant