Repository navigation
feat(web): float device streams over chat - #408
Conversation
Port the upstream floating device stream UX while preserving persisted browser previews, SSH host identity, and thread-scoped cleanup.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
added
vouch:trusted
There was a problem hiding this comment.
🟡 Changes recommended
Critical and moderate behavior and accessibility issues remain unresolved.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Extends the web mini-player to support browser tabs and host-qualified device streams with persistence, lifecycle handling, and documentation.
Changes:
- Adds source-aware mini-player state, layout, resizing, and migration.
- Adds device floating, promotion, cleanup, and automation behavior.
- Updates settings, documentation, and coverage.
File summaries
| File | Description |
|---|---|
docs/user/devices.md |
Documents device floating controls and preferences. |
apps/web/src/previewMiniPlayerStore.ts |
Adds source-aware state and legacy migration. |
apps/web/src/previewMiniPlayerStore.test.ts |
Tests persisted mini-player behavior. |
apps/web/src/components/settings/SettingsPanels.tsx |
Updates floating-preview preference text. |
apps/web/src/components/preview/ThreadPreviewMiniPlayer.tsx |
Renders browser and device mini-players. |
apps/web/src/components/preview/ThreadPreviewMiniPlayer.test.tsx |
Tests mini-player rendering. |
apps/web/src/components/preview/PreviewView.tsx |
Integrates browser floating controls. |
apps/web/src/components/preview/PreviewView.test.tsx |
Tests browser floating behavior. |
apps/web/src/components/preview/previewMiniPlayerLayout.ts |
Calculates sizing and placement. |
apps/web/src/components/preview/previewMiniPlayerLayout.test.ts |
Tests layout calculations. |
apps/web/src/components/preview/PreviewAutomationHosts.tsx |
Applies automated presentation behavior. |
apps/web/src/components/device/DevicePanel.tsx |
Adds manual device floating. |
apps/web/src/components/ChatView.tsx |
Handles device presentation, cleanup, and promotion. |
apps/web/src/components/ChatView.test.ts |
Tests source-aware visibility rules. |
apps/web/src/components/ChatView.browser.tsx |
Covers browser mini-player integration. |
Review details
Suppressed comments (5)
apps/web/src/components/ChatView.tsx:2092
- This second device-promotion path also strips
serverEpochfrom the floating source. The resulting right-panel target is always treated as legacy/recoverable byDevicePanel, so a missing session can trigger an unnecessarylist/openand restart a device that ended. Preserve the current device server epoch here as well, and include it in this callback's dependency list.
state.openDevice(activeThreadRef, {
hostId: floatingPreview.source.hostId,
deviceId: floatingPreview.source.deviceId,
platform: floatingPreview.source.platform,
name: floatingPreview.source.name,
});
apps/web/src/components/ChatView.tsx:2001
- When this effect first sees a loaded thread,
previousis undefined, so the early return skips every existing session; a device opened before the thread mounted is never auto-presented. It also skips all new sessions in sheet layouts instead of applying the preference while retaining the existing right-panel sheet behavior. Initialize the baseline separately and only use the sheet condition to choose panel vs. float presentation.
if (!previous || shouldUseRightPanelSheet) return;
apps/web/src/components/preview/ThreadPreviewMiniPlayer.tsx:395
- On coarse-pointer devices this pill remains
opacity-0 pointer-events-none: the only visible 12px dot is decorative and has no pointer handler, while hover is unavailable. The documented Open in right panel and Close controls are therefore unreachable by touch; make the pill visible and interactive under the existingpointer-coarsevariant, or provide a tap-to-reveal control.
className="pointer-events-none absolute right-0 top-0 flex h-8 cursor-grab items-center gap-0.5 rounded-lg border border-border/80 bg-popover/92 p-0.5 opacity-0 shadow-lg/20 backdrop-blur-xl transition-opacity group-hover:pointer-events-auto group-hover:opacity-100 group-focus-within:pointer-events-auto group-focus-within:opacity-100 active:cursor-grabbing"
apps/web/src/components/preview/ThreadPreviewMiniPlayer.tsx:445
- These resize affordances are presentation-only
divs with pointer handlers, so keyboard and screen-reader users have no focusable or labeled way to resize the player. This also removes the previous labeled resize button; use focusable controls with an accessible resize interaction instead of exposing only pointer hit zones.
<div
key={direction}
role="presentation"
data-preview-mini-player-resize={direction}
className={cn("pointer-events-auto absolute z-[49] touch-none", className)}
apps/web/src/previewMiniPlayerStore.test.ts:121
- The compatibility branch that migrates legacy
{ tabId, size }persistence is not covered here: this fixture now uses only the newsource/widthshape. Add a legacy persisted entry and assert that it normalizes to a browser source with the old width so this migration does not regress unnoticed.
source: browserMiniPlayerSource("tab-a"),
position: { x: 20, y: 30 },
width: 360,
- Files reviewed: 15/15 changed files
- Comments generated: 5
- Review effort level: Lite (auto)
Note
Copilot is running an experiment and ran this review at Lite.
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| usePreviewMiniPlayerStore | ||
| .getState() | ||
| .open(THREAD_REF, browserMiniPlayerSource("preview-browser-test")); |
| usePreviewMiniPlayerStore | ||
| .getState() | ||
| .open(activeThreadRef, browserMiniPlayerSource(activeSurface.resourceId)); |
| const openInPanel = () => { | ||
| usePreviewMiniPlayerStore.getState().close(threadRef); | ||
| useRightPanelStore.getState().openDevice(threadRef, { | ||
| hostId: source.hostId, | ||
| deviceId: source.deviceId, | ||
| platform: source.platform, | ||
| name: source.name, | ||
| }); | ||
| }; |
| const source = | ||
| parseSource(candidate.source) ?? | ||
| (typeof candidate.tabId === "string" && candidate.tabId.length > 0 | ||
| ? browserMiniPlayerSource(candidate.tabId) | ||
| : undefined); | ||
| const position = parsePosition(candidate.position); | ||
| const size = parseSize(candidate.size); | ||
| if (position === undefined || size === undefined) continue; | ||
| byThreadKey[threadKey] = { tabId: candidate.tabId, position, size }; | ||
| const width = parseWidth( | ||
| candidate.width ?? (isRecord(candidate.size) ? candidate.size.width : null), | ||
| ); |
| to the thread. Mobile clients show device activity in the thread timeline. Agents drive the device through the `agent-device` command line. T3 | ||
| When an agent opens a device, it floats over the chat in web and desktop clients | ||
| connected to the thread, the same way an agent-driven browser does. Turn off | ||
| **Auto-show floating preview** in **Settings → Devices** to open a right-panel |
Updated floating device feature validationRetested the unchanged PR head Scenario exercised on the local booted iPhone 17 Pro simulator (
Diagnostics: no application console errors. The only non-feature failed requests were expected tracing cancellations; stream transitions produced expected aborted requests when switching surfaces, while active MJPEG/AVCC endpoints returned Actual feature evidence
The native agent-driven invocation remains unverified because the configured Codex app-server provider exits with code 1 before executing |
|
Updated with commit 24a10cb: reconcile already-active device sessions on first load, retain device server epochs and recover floating sessions after restart, and restrict browser source fitting to fixed/device viewports. Added focused regression coverage. Validation: 61 focused tests passed; pnpm fmt:check, pnpm lint, and pnpm typecheck passed (existing warnings only). Live feature validation remains blocked: the prescribed isolated pnpm dev command exits with Unrecognized flag: --filter before serving the app; no new browser captures were produced. |
Summary
Port the focused floating device stream UX from upstream
pingdotgg/t3code#11285without cherry-picking unrelated infrastructure.{ tabId, size }shape.Upstream provenance
Adapted from merged upstream PR pingdotgg/t3code#11285, primarily commits
ec0481ed96999d873ff54b1e7221346195cbad77and its follow-up layout/pending-device fixes. The fork-specific port retains the SSH host/environment identity and Device Hub APIs already present onmain.Validation
pnpm fmt:check— passed.pnpm lint— passed with existing repository warnings only.pnpm typecheck— passed with existing repository suggestions only.pnpm test— passed: web 184 files / 1,878 tests; full workspace completed successfully.preview_statusfirst; the preview webview timed out and the web client reports this preview surface as desktop-only. Playwright fallback validated the authenticated app, device settings, local Device Hub access, source-store cleanup, and browser UI state. No SSH hardware was claimed or exercised.Tested revision
dd02ab8a5905eeb4eee90dcd96fcc01e46557455Limitations
Native simulator interaction through an agent-opened device session remains unverified in this environment because the configured Codex provider could not start. The implementation reuses the existing DeviceStreamView reconnect/input path and is covered by source/layout/store/component tests; no remote hardware validation was performed.
Feature captures
PR head at upload:
dd02ab8a5905eeb4eee90dcd96fcc01e46557455. See the testing notes for exercised behavior and limitations; uploading media is not a test result.page@e2d0b74c098bc49863e1f6c358c16926.webm
page@e2d0b74c098bc49863e1f6c358c16926.webm