perf(chat): read the conversation when its transcript changes, not every 2 s

CHANGELOG.md

@@ -7,6 +7,18 @@ between releases do not reach them. Remote-PC runtime bundles are versioned sepa
 
 ## [Unreleased]
 
+### Changed
+- Chat updates follow native transcript changes pushed through the existing WebSocket.
+  Only visible Chat views subscribe; Terminal views and hidden tabs release conversation
+  monitoring. Continuous pushes coalesce into at most one newest-page refresh every 2 seconds,
+  with a 10-second fallback; without pushes (an older remote PC, or a tab that let go of its pane)
+  Chat reads every 2 seconds as before. Initial display, sends, reconnect and returning to the tab refresh
+  immediately without overlapping reads or dropping held history. A Chat that shows the
+  terminal's text because no native transcript exists yet rereads on the pane's output instead.
+  Conversation responses use negotiated gzip locally, and through a remote PC once its bridge
+  runs a bundle with this change, while retaining conditional reads and pagination.
+  Approval-card polling remains independent.
+
 ### Fixed
 - A request that carries a session cookie but states no `Origin` is now read as cross-site rather
   than same-origin, so a header that went missing is no longer trusted as though a browser had

docs/chat-mode-audit.md

@@ -34,8 +34,8 @@ changes, and Claude assistant-message models. Missing effort stays `null`; it is
 never inferred from reasoning text. Model and effort are cached with the turns.
 
 The composer status line displays the model and `Reasoning <level>` on desktop
-and mobile, or `Reasoning —` when unavailable. Existing conversation polling
-refreshes these labels even without a new message. Metadata is tagged with its
+and mobile, or `Reasoning —` when unavailable. A conversation read after each native
+transcript change refreshes these labels even without a new message. Metadata is tagged with its
 pane so a pane switch cannot display another conversation's settings; scrollback
 fallback clears it. The live Codex API returned the configured model and `xhigh`
 effort after the change. Refresh the page to load the rebuilt client.

docs/development.md

@@ -89,6 +89,10 @@ Tests run against a herdr session of their own, `herdr-web-ui-test`. The first r
 
 Browser checks look for Chrome at `/opt/google/chrome/chrome`; set `CHROME_PATH` otherwise. After a herdr upgrade, refresh the generated wire types with `bun run generate:types --refresh` (and `--check` to verify).
 
+`HEADED=1 bun scripts/chat-browser-qa.ts` opens its owned native-transcript fixture in a
+visible Chrome window. Its detected Codex fixture process keeps the rollout open; no
+real model is invoked by this script. Set `CHROME_PATH` to the macOS Chrome binary there.
+
 For isolated phone viewer and keyboard layout regressions (no herdr session; the demo runner builds the real client locally):
 
 ```bash
@@ -106,6 +110,118 @@ bun scripts/font-swap-demo-regression.ts      # the app's faces arriving late on
 
 `FILE_VIEWER_CASE=landscape-notch` selects a viewer case; `FILE_VIEWER_CSS=/path/to/before.css` compares another stylesheet. These checks use Chromium mobile emulation and synthetic safe-area/keyboard geometry; they cannot verify actual iOS Safari keyboard dismissal or notch insets. The existing `bun scripts/file-viewer-regression.ts` separately checks history with an owned herdr pane. The original `scripts/mobile-viewport-regression.ts` exports `checkMobileViewport` for the real-app `bun run test:ui` suite; it also checks the command palette and xterm focus transitions. The demo runners build the real client into a temporary directory, inject the committed fictional-session transport and serve it only on loopback; they do not use a live herdr session or download website media. Run on its own, each builds the client itself (`scripts/demo-build.ts`); the browser lane builds it once and names the directory in `HERDR_DEMO_BUILD`, and each runner copies that instead. They exercise real-app viewport and alert geometry, but not live herdr connectivity.
 
+## Conversation refresh
+
+The chat reads `/api/pane/conversation` after a `conversation-changed` WebSocket frame.
+The frame contains only `pane_id` and an opaque `signature`; REST remains the authoritative
+paged conversation, including metadata and history identity.
+
+A mounted, visible Chat enables `conversation-watch {pane_id, enabled:true}` on its existing
+machine-bound WebSocket, after attaching. Bridges advertise the `"conversation-watch"` feature;
+older bridges receive no unknown control frame, and their visible Chat reads every 2 s as before.
+The client retains current interest through reconnect, replaying it after attachment and
+capability discovery. Hiding the page, leaving Chat or unmounting disables interest. The bridge
+drops interest with the attachment, and the client asks again with the pane's next attach: a
+tab that let go of its pane while out of use (Settings → Use alongside herdr's own window) has
+no pushes while it only watches, so it reads every 2 s, and gets pushes back when it attaches.
+Observers may subscribe without gaining input or resize authority.
+
+The bridge shares one native-file monitor per interested attached pane and one 250 ms stat
+timer. Terminal-only and hidden clients do not keep a conversation monitor alive. The monitor
+resolves through existing provider/process evidence, checks file identity, size and timestamps
+without parsing turns, and rechecks session resolution every 5 s (2 s while a transcript is
+missing). The last interested client, failed/ended attachment and server stop release it.
+
+Newest-page refreshes are single-flight. Push-triggered reads start at least 2 s apart using
+monotonic elapsed time, with one trailing refresh at a fixed deadline: continuous writes cannot
+keep postponing it. A push after idle reads immediately. Initial display, explicit sends,
+status/history repairs, reconnect and visibility return bypass that delay but not serialization.
+Gap fills run inside the newest read. An older page starts once the newest read in flight has
+settled, so a history change found there drops it first, but it never holds the next newest read
+back: a clear or a new turn shows while a slow older page loads. A visible Chat with pushes retains a 10 s
+backstop for lost notifications; without pushes (an older bridge, a released pane) the backstop
+is the former 2 s poll. Hidden Chat starts no new automatic reads or queued
+pagination work; returning preserves held history and scroll. Pending work never follows a
+pane or machine switch.
+
+A Chat showing the scrollback stand-in (no native transcript yet, or none recognized) has no
+file for the monitor to watch. While it is visible, the pane's own `pty-data` (a mirror's screen
+frames on Windows) invalidates it through the same coordinator, so continuous output rereads
+at most every 2 s and a quiet pane is not read. A Chat showing a native transcript ignores
+terminal output.
+
+Conversation responses negotiate gzip, including through the remote-PC HTTP proxy.
+Encoding-independent weak ETags preserve conditional reads; 304 has no body and responses
+vary on `Accept-Encoding`. No other endpoint is compressed by this change.
+Prompt polling remains separate at 2 s: approval cards and suggestions can change on the
+terminal screen without a transcript write or agent-status transition.
+
+### Initial push + gzip measurement
+
+On macOS, headed Chrome read an owned native omp fixture with 22 prompt/answer pairs
+(initial JSON about 580.7 KB). Five assistant appends ran in a 30 s interval with the agent
+status held constant. Initial loading was excluded. The same loopback fixture was measured
+before and after; this is not a bandwidth estimate for arbitrary conversations.
+
+These measurements predate explicit visible-Chat subscriptions and the 2 s push-read bound.
+They are evidence for the initial push/gzip implementation, not a remeasurement of the current
+refresh policy.
+
+| Metric | Before: 2 s polling | After: push + gzip |
+| --- | ---: | ---: |
+| Write-to-visible latency (ms) | 1659, 666, 684, 697, 710 | 233, 292, 55, 102, 137 |
+| Mean latency | 883.2 ms | 163.8 ms |
+| Conversation requests / 30 s | 15 (5 × 200, 10 × 304) | 5 (200) |
+| Encoded HTTP body bytes | 2,903,810 | 324,544 |
+| New conversation WS payload bytes | 0 | 430 |
+| Body + invalidation payload bytes | 2,903,810 | 324,974 |
+
+Mean latency fell 81.5% and the measured payload fell 88.8%. HTTP/TLS headers, terminal
+traffic and prompt requests are excluded. The deliberately repetitive fixture compressed
+to about 11.2% of its identity response; real tool output and conversations will differ.
+
+### Visible-Chat subscription and bounded-refresh resource measurement
+
+The current policy was compared with upstream 2 s polling on macOS in headed Chrome,
+using owned native Codex fixtures initially about 60 KB each. Each row is a paired
+20 s sample, excluding initial loading, with fresh bridge processes and the same
+16 owned workspaces present for both versions. Both versions retain the existing
+WebSocket connections; "push" below means WS invalidation followed by REST reads,
+not conversation bodies delivered over WS.
+
+CPU is cumulative CPU time of the Bun bridge during the sample; RSS is its sampled
+mean. Chrome, herdr, PTY and resolver subprocess CPU is excluded. Hidden visibility
+was simulated; these are not physical-phone or long-running leak measurements.
+Quota requests were blocked equally in both versions.
+
+| Scenario | Conversation REST reads: polling → push | Bun CPU ms: polling → push | Mean Bun RSS MiB: polling → push |
+| --- | ---: | ---: | ---: |
+| One idle Chat | 9 → 2 | 119 → 135 | 54.4 → 60.6 |
+| Eight idle Chats, same pane | 78 → 15 | 381 → 275 | 58.2 → 68.4 |
+| Eight idle Chats, distinct panes | 78 → 16 | 351 → 343 | 70.4 → 71.4 |
+| Eight distinct Chats, writes every 4 s | 78 → 39 | 370 → 333 | 70.4 → 73.2 |
+| Eight distinct Terminal views | 0 → 0 | 108 → 112 | 58.7 → 58.3 |
+| Eight hidden Chats | 0 → 0 | 82 → 100 | 67.6 → 67.8 |
+| Eight distinct Chats, writes about every 100 ms | 78 → 80 | 416 → 485 | 73.3 → 72.7 |
+| Eight ordinary-shell Terminal views | 0 → 0 | 143 → 150 | 57.2 → 57.1 |
+
+The high-frequency run wrote 197 updates per pane: 656 WS invalidations produced
+80 REST reads across eight views, and every view eventually rendered its final
+update. Its bridge CPU remained about 16.5% above polling; bounded reads do not
+eliminate server monitor work. Resolver subprocess launches were 78 → 110 in that
+sample, so bridge CPU alone must not be presented as total system CPU.
+
+Terminal-only, hidden and ordinary-shell Terminal samples had zero conversation
+monitors, file-stat checks, transcript resolutions, invalidations and REST reads.
+Eight views kept eight live sockets in both versions; same-pane Chats shared one
+monitor. Leaving Chat released interest, reconnect restored it, and returning from
+hidden rendered an appended marker in under 2 s. Closing seven same-pane views
+retained the monitor; closing the last view removed all monitors, the monitor timer
+and owned PTY children. No browser errors occurred in these samples.
+
+These resource results do not remeasure the latency or compressed payload figures
+in the historical table above.
+
 ## README media
 
 `bun run build && bun scripts/readme-media/capture.ts` regenerates the stills and demos in `docs/screenshots/` from a staged, fictional session in its own herdr session (`herdr-web-ui-demo`). Pass `shots` or `video` to redo only one of them. It needs ffmpeg.

scripts/chat-browser-qa.ts

@@ -3,12 +3,12 @@
 import "./test-herdr.ts"; // a herdr session of its own: nothing shows in the user's
 import assert from "node:assert/strict";
 import { Database } from "bun:sqlite";
-import { copyFileSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { chmodSync, copyFileSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
 import { tmpdir } from "node:os";
 import { join } from "node:path";
 import { chromium } from "playwright-core";
 import { createServer } from "../server/index.ts";
-import { herdrRpc, workspaceClose, workspaceCreate } from "../server/herdr/client.ts";
+import { herdrRpc, sessionSnapshot, workspaceClose, workspaceCreate } from "../server/herdr/client.ts";
 
 const root = mkdtempSync(join(tmpdir(), "herdr-web-ui-chat-browser-"));
 const codexHome = join(root, "codex");
@@ -41,19 +41,34 @@ try {
   add({ type: "function_call_output", call_id: "c2", output: "Process exited with code 1\n1 fail" }, 5);
   add(message("assistant", answer, "final_answer"), 8);
   persist();
+  // A detected fixture process keeps native descriptor evidence for the rollout;
+  // the ignored path field of a manual lifecycle report is not a session binding.
+  mkdirSync(join(root, "bin"));
+  const executable = join(root, "bin", "codex");
+  writeFileSync(executable, `#!/bin/sh\nexec 3<"$1"\nprintf '%s\\n' '${answer}'\nwhile IFS= read -r line; do :; done\n`);
+  chmodSync(executable, 0o755);
   const created = await workspaceCreate({ cwd: root, label: "herdr-web-ui-test-chat-browser" });
   workspaceId = created.workspace.workspace_id;
   const paneId = created.root_pane.pane_id;
-  await herdrRpc("pane.report_agent", { pane_id: paneId, source: "manual", agent: "codex", state: "idle", agent_session_path: rollout });
-  await herdrRpc("pane.send_text", { pane_id: paneId, text: `printf '%s\\n' '${answer}'\n` });
+  await herdrRpc("pane.send_text", { pane_id: paneId, text: `${executable} ${rollout}\n` });
+  // Real herdr process detection runs outside this browser check's clock.
+  const deadline = Date.now() + 10_000;
+  while ((await sessionSnapshot()).panes.find((pane) => pane.pane_id === paneId)?.agent !== "codex") {
+    if (Date.now() >= deadline) throw new Error("owned Codex fixture process was not detected");
+    await Bun.sleep(50);
+  }
+  await herdrRpc("pane.report_agent", { pane_id: paneId, source: "manual", agent: "codex", state: "idle" });
   // tailscaleOwner null: the plain-HTTP page below reaches the server under a name that is not
   // loopback, which a PC running Tailscale would otherwise send to pairing
   server = createServer({ port: 0, hostname: "127.0.0.1", token: "", stateDir: join(root, "push"), codexHome, tailscaleOwner: null });
-  browser = await chromium.launch({ executablePath: process.env["CHROME_PATH"] ?? "/opt/google/chrome/chrome", headless: true, args: ["--no-sandbox", "--host-resolver-rules=MAP clipboard.test 127.0.0.1", "--no-proxy-server"] });
+  browser = await chromium.launch({ executablePath: process.env["CHROME_PATH"] ?? "/opt/google/chrome/chrome", headless: process.env["HEADED"] !== "1", args: ["--no-sandbox", "--host-resolver-rules=MAP clipboard.test 127.0.0.1", "--no-proxy-server"] });
   const context = await browser.newContext({ viewport: { width: 1280, height: 800 }, locale: "en-US" });
   const page = await context.newPage();
   // a pane opens its terminal the first time; this QA is about the chat lens
-  await page.addInitScript((id) => localStorage.setItem(`herdr-web-ui:view:${id}`, "chat"), paneId);
+  await page.addInitScript((id) => {
+    localStorage.setItem("herdr-web-ui:settings", JSON.stringify({ language: "en" }));
+    localStorage.setItem(`herdr-web-ui:view:${id}`, "chat");
+  }, paneId);
   const errors: string[] = [];
   page.on("pageerror", (error) => errors.push(error.message));
   page.setDefaultTimeout(10_000);
@@ -85,7 +100,7 @@ try {
     return hidden.every((item) => item !== null && item.textContent !== "" && item.getBoundingClientRect().width <= 1);
   }), true, "the agent's name, every pane status word and the reasoning sentence are read, not drawn");
   const work = log.locator(".work-block-head");
-  const report = (state: "working" | "idle") => herdrRpc("pane.report_agent", { pane_id: paneId, source: "manual", agent: "codex", state, agent_session_path: rollout });
+  const report = (state: "working" | "idle") => herdrRpc("pane.report_agent", { pane_id: paneId, source: "manual", agent: "codex", state });
   const head = (expanded: boolean, title: string) => log.locator(`.work-block-head[aria-expanded="${expanded}"]`).filter({ hasText: title });
   await head(false, "Worked for 7s").waitFor();
   await report("working");
@@ -160,19 +175,26 @@ try {
   assert.equal(await nativeImage.count(), 1);
   console.log("PASS native Codex image-only turn loads its pane-scoped thumbnail");
 
-  const gate = new Promise<void>((resolve) => { release = resolve; });
+  const { promise: gate, resolve } = Promise.withResolvers<void>();
+  release = resolve;
   let requests = 0;
   await page.route("**/api/pane/conversation?*", async (route) => { requests += 1; await gate; await route.continue(); });
   await page.getByRole("textbox", { name: "Message", exact: true }).fill("true");
   await page.getByRole("button", { name: "Send message", exact: true }).click();
-  await page.waitForTimeout(250);
+  // Observe the real browser's send and timer path; this bounded negative window
+  // catches the old overlapping 2-second refresh without faking herdr/Chromium clocks.
+  const readDeadline = Date.now() + 10_000;
+  while (requests === 0) {
+    if (Date.now() >= readDeadline) throw new Error("send refresh did not begin");
+    await Bun.sleep(25);
+  }
   assert.equal(await log.getByText(answer, { exact: true }).count(), 1, "send refresh must not blank existing history");
   const beforeWait = requests;
   await page.waitForTimeout(2500);
-  assert.equal(requests, beforeWait, "slow polls must not overlap");
+  assert.equal(requests, beforeWait, "slow refreshes must not overlap");
   release();
   await page.unrouteAll({ behavior: "wait" });
-  console.log("PASS send refresh preserves history and slow polling does not overlap");
+  console.log("PASS send refresh preserves history and slow refreshes do not overlap");
 
   await page.setViewportSize({ width: 390, height: 844 });
   await page.locator("#workspace-drawer").waitFor({ state: "hidden" });