Skip to content

Remote Mobile Companion: control agents from your phone (LAN + tunnel) - #430

Merged
Juliusolsson05 merged 15 commits into
mainfrom
feat/remote-mobile-companion
Jul 6, 2026
Merged

Juliusolsson05 merged 15 commits into
mainfrom
feat/remote-mobile-companion

Conversation

@Juliusolsson05

Copy link
Copy Markdown
Owner

Closes the build phases of #420.

What

Watch every session, send prompts, and answer permission dialogs from a phone browser — driving the real agents. LAN by default (zero external dependencies); optional cloudflared quick tunnel for use away from the network.

How it's built

  • Phase 0 — SessionFeed contract (src/shared/sessionFeed/): the single seam between "UI wants session I/O" and "where it comes from". Desktop = IpcSessionFeed (pure pass-through to the preload bridge, zero behavior change); phone = WebSocketSessionFeed. The renderer now provably runs against a fake feed with no Electron.
  • Phase 1 — LAN companion (src/main/remote/, src/remote-client/): QR pairing → revocable per-device HMAC tokens (registry = the kill switch), WS server with token checks at upgrade AND per message, and a v1 wire protocol where the zod union IS the capability ceiling — shell exec, session lifecycle, and raw input are unrepresentable. Phone client: scan-to-pair, live session list, TUI-text session view, condition tap-targets, composer.
  • Phase 2 — tunnel: cloudflared pinned in third_party/ (tmux/mitmproxy convention, real hashes), loopback-only tunnel transport, LAN/Tunnel switcher. A missing binary fails the tunnel toggle loudly; LAN never touches cloudflared.

Isolation (for future contributors)

One-way dependency wall: core never imports from src/main/remote/ or src/remote-client/; deleting both leaves core compiling. Sanctioned holes only: the shared contract, one construction line in main/index.ts, one IPC registration, one preload spread.

Review

A 28-agent high-effort review swept the full diff; all 10 confirmed findings fixed in 8a025f0 (pre-enable session/prompt visibility via new SessionManager snapshot caches, provider prompt-delivery discipline for phone prompts, removed-without-exit propagation, enable-race serialization, VPN-aware LAN IP ranking, terminal-session exclusion, script copy-paste strings).

Verification

  • 258/258 tests (76 covering the remote subsystem, incl. real-socket client↔server integration suites)
  • tsc clean on both projects; phone bundle builds (npm run client:build)

🤖 Generated with Claude Code

Juliusolsson05 and others added 15 commits July 6, 2026 21:02
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…o shared

The SessionFeed interface is the single seam between 'the UI wants live
session I/O' and 'where it physically comes from' — desktop IPC today,
the remote mobile client's WebSocket transport in phase 1. Event payload
types move from @preload/api/types to @shared/sessionFeed/types (preload
re-exports byte-for-byte) so the contract is implementable without
Electron.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Zero-logic delegation to the flat window.api bridge — the desktop's
feed-selection point. Lazy window.api reads keep the module import-safe
before contextBridge assembly and under test stubs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…double

Context makes transport a mount-time decision (desktop IPC vs remote WS vs
test fake) instead of an import-time coupling. The fake is a real minimal
implementation, not a spy bundle, so passing tests prove contract adherence.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 9 session-event listeners now come from the SessionFeed param instead
of window.api; desktop passes ipcSessionFeed (module const — identity-
stable, which the effect dep array requires). Desktop-only side channels
(ghostAppend, gitWorktrees, feed-debug, perf) intentionally stay on
window.api. Harness test proves the hub folds events from a fake feed
with no Electron present.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…top provider

Composer submit (TileLeaf), condition resolution (TileLeaf outlet +
AskUserQuestionRow), terminal-nav forwarding, and terminal-sink dictation
now use the injected feed. SessionFeedProvider mounts at the app root in
main.tsx — the desktop's single feed-selection point — and useWorkspace
reads it from context. Raw-terminal leaves (TerminalLeaf,
AgentTerminalLeaf, AgentInlineTerminal) stay on window.api on purpose:
raw PTY byte streams are desktop-only and outside the v1 remote scope.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Auth: per-install HMAC secret (0o600 under STATE_DIR/remote), persisted
device registry as the revocation authority, and QR pairing with
single-use 5-minute codes minting stateless HMAC device tokens.

Protocol: zod schemas where the inbound discriminated union IS the v1
allow-list — shell exec, session lifecycle, provider switch, and raw
input are unrepresentable, not merely denied. scope.test.ts pins both
directions so widening the protocol forces a visible test edit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SessionFeedSource is a second parallel subscriber on SessionManager
(forwarder untouched) with its own per-session jsonl coalescer; raw PTY
channels are deliberately never tapped so remote clients cannot observe
terminal bytes. RemoteTransport is the reachability seam that makes the
phase-2 cloudflared tunnel a transport swap, not a server change;
LanTransport binds 0.0.0.0 and reports one concrete LAN IPv4 for the QR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… host

Serves the remote-client bundle (placeholder until built), redeems
pairing codes over POST /pair, and hosts the WS endpoint with the
three-layer posture: token at upgrade, token re-verified per message
(revocation bites mid-session), and the v1 schema as the capability
ceiling. pty condition actions apply only when they exactly match an
action the live condition snapshot offered — the phone picks from the
provider's menu, it never composes keystrokes. Late joiners get cached
screen/conditions/process-state so idle sessions aren't blank panes.
Integration-tested over real sockets end to end (11 cases).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…C, RemotePanel

RemoteController owns the enable → pair → revoke → disable lifecycle:
durable state (secret, registry) survives toggles, live state (feed tap,
server) is torn down on disable so the subsystem costs zero when off.
remote:* IPC handlers are one-liners onto it (sanctioned boundary hole),
with a status push channel mirroring the caffeinate pattern. RemotePanel
(command palette → 'Remote Control') renders the pairing QR locally via
the qrcode package — the one-time code rides the URL fragment so it never
appears in HTTP logs. Wired into main lifecycle: constructed default-off,
disposed on quit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…uild target

The phone implements the same SessionFeed contract the desktop does, over
WS: session-event frames fan out to the standard listener shape; commands
become protocol frames correlated by client-minted ids; reconnect is
capped-backoff and self-healing because the server replays list + cached
state on every connect. sendInput translates exactly the three byte shapes
the v1 scope covers (submit/interrupt/bracketed paste) and rejects raw
bytes — widening that means widening the wire protocol first.

UI: scan QR → auto-pair via #code fragment (scrubbed after redeem) →
session list with live activity dots → session view rendering the
provider's own TUI text with condition prompts as tap targets and a
paste-then-submit composer. Plain-Vite build target (out/remote-client)
served by RemoteServer when present; type-checked under tsconfig.web so
client drift can't ship silently. Client↔server drift is pinned by a
7-case integration suite running the real feed against the real server.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…groundwork)

manifest.json pins release 2026.6.1 with archive AND extracted-binary
sha256 for both darwin arches (hashes computed from real downloads);
fetch/verify scripts are mechanical siblings of the tmux ones (same
double-pinning discipline, --version instead of -V for the native sanity
check). Resolver + copy-packaged-resources wired. The remote feature does
NOT depend on this binary: LAN mode never touches cloudflared, and a
missing binary only disables the (upcoming) tunnel toggle with a visible
error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…se 2)

CloudflaredTunnel binds loopback only (with a public tunnel up, ALSO
listening on the LAN would double the exposed surface) and parses the
ephemeral trycloudflare URL from the child's output; teardown never leaks
a half-started state. Tested against a fake shell-script binary so CI
needs no network. enable(mode) switches transports cleanly — sockets drop,
pairing survives (durable state). Tunnel failure is LOUD, never a silent
LAN fallback: the user asked for internet reachability and a LAN URL would
misrepresent what the QR exposes. LAN remains the default and never
touches cloudflared — the feature fully works locally without the binary
(pinned by the new controller test).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Headline-flow fixes:
- SessionManager now caches last screen/conditions snapshots per session
  (cleared on removal) with getters; SessionFeedSource seeds from
  manager.list() and RemoteServer primes its late-joiner caches from the
  snapshot getters — agents started (or blocked on permission prompts)
  BEFORE remote was enabled are now fully visible and answerable from the
  phone, which was the feature's advertised scenario.
- send-prompt routes EVERY kind through deliverPromptToAgent so the
  provider's own prompt-delivery module runs (Claude paste-absorption →
  Enter, Codex readiness gate); the phone's separate submit frame is gone.
  Bare bracketed-paste + racing '\r' was the exact swallowed-Enter bug the
  desktop promptDelivery modules document.
- 'removed' is now a broadcast channel: removed-without-exit paths (tmux
  detach, spawn rollback) prune the phone list and the server caches
  instead of pinning dead sessions forever.
- RemoteController transitions run on a strict FIFO chain (mode-ignoring
  coalescing + double-click double-server races retired) and doEnable
  builds the transport BEFORE the feed tap, so a failing tunnel enable no
  longer leaks manager listeners per retry.
- pickLanIpv4 ranks physical interfaces over VPN/virtual ones (utun,
  docker, vmnet…) so the pairing QR stops advertising unreachable
  addresses when a VPN is up.
- Terminal sessions are excluded from the remote feed entirely: they can
  never render output remotely, so listing them was an invisible
  command-execution surface.

Cleanups from the same review: cloudflared script error strings no longer
name tmux; CloudflaredTunnel detaches its output listeners once settled;
registry touch persistence throttled to 1/min (was every frame); dead
'pong' frame and unused device-name persistence removed; @types/qrcode
moved to devDependencies.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant