feat: wire boxd into the app (#356)

AGENTS.md

@@ -38,6 +38,16 @@ Go uses `testing` with `*_test.go` files and `Test…` functions. Swift uses XCT
 
 Use concise, descriptive commits. History commonly uses `fix(macapp): …`, `refactor(macapp): …`, and `docs(…): …`; plain imperative subjects also occur. PRs should explain behavior changes, link issues (`Fixes #177`), report validation and skipped checks, and include screenshots for visible UI changes.
 
+## Reviews & Reported Bugs
+
+A finding from a review, a reviewer bot or an agent is not a bug until it is checked against how the app is actually used, or could plausibly be used. Before fixing or filing it, answer three questions:
+
+- **What triggers it?** Name the real sequence: what the user does, what the environment does, and what timing it needs.
+- **How likely is that?** It needs a realistic path in ordinary use, or a failure that does happen (an outage, a quit mid-operation, a corrupt config). A race that needs a click inside one actor hop, or input the code never produces, does not qualify.
+- **What does it cost when it happens?** Lost or corrupted data, a paid resource left running, a broken or misleading UI, or nothing that persists.
+
+Fix it only when it is likely enough to happen, or costly enough when it does. Otherwise reply on the thread with that reasoning and close it. File an issue only when a future change could make it reachable. Comment wording, test-timing hypotheticals and style suggestions never justify another review round.
+
 ## Architecture & Configuration
 
 Preserve the CLI `--json` contract; breaking changes require a `schema_version` bump in `cmd/json.go`. Read `CONTRIBUTING.md` and [repository notes](docs/repository-notes.md) for architecture and operations. For app work, also follow [macapp/AGENTS.md](macapp/AGENTS.md). Never use personal workrooms as destructive test fixtures.

TODOS.md

@@ -7,99 +7,30 @@
 
 ## P0 — before the next release
 
-## P1 — before GA
-
-### A second Mac can find a remote workroom and reach its host (macapp, Codaset) — #255 blocker
-
-**What:** Let a Mac that did not create a remote workroom discover it and get an ssh key its host
-accepts.
-
-**Why:** #255 (cross-machine reattach) cannot be tested on a real host until this exists. Today a
-remote workroom is listed only in the creating Mac's CLI config, and its host accepts only that
-Mac's key (`RemoteWorkrooms.swift` mints its `id_ed25519` once per Mac per build). Container
-workrooms on one Mac's Docker are unreachable from another Mac at all, so the live test runs on
-boxd. Design: `docs/designs/oq8-cross-machine-reattach.md`, premise 10.
-
-**How to start:** The Codaset broker already keys grants by `workroomID` per user (#250, #251).
-List the signed-in user's grants to discover workrooms, and enrol this Mac's public key on the host
-through the broker rather than by hand.
-
-**Depends on:** #250 and #251 (done). Blocks #255's live acceptance test.
+### `AgentWakefulnessTests.testAReplyCrossingAKeepDoesNotResurrectTheCard` flakes (macapp) — #216 test
 
-**Priority:** P1 (chosen in the #255 eng review, 2026-10-05)
+**What:** Find why the crossing-`keep` test sometimes fails and fix the test or the race it exposes.
 
-### Before wiring boxd into the app: a wake-on-connect resets the awake ceiling (wr-agent) — #257 review
+**Why:** It failed once in a targeted `/ship` QA run on `feat/boxd-in-app` (2026-10-07, 2 of 80
+`AgentWakefulnessTests` red: "the failure brought the retry back", `XCTAssertTrue failed` at the
+retry-flag assert), then passed in every full suite after it (3267/0, four runs). The test came with
+#216 on master; the boxd branch didn't touch the keep/retry path. A flake in the prompt card's retry
+logic can hide a real resurrection bug, which is what the test exists to catch.
 
-**What:** Decide what a resume does to the awake ceiling once a remote host can sleep, before
-`BoxdHostDriver` is wired into `RemoteHosts`.
+**How to start:** Loop it alone (`make app-test "APP_TEST_FLAGS=-only-testing:WorkroomAppTests/AgentWakefulnessTests/testAReplyCrossingAKeepDoesNotResurrectTheCard"`)
+under CPU load until it fails. Then check whether the crossing reply and the failed `keep` can land in
+either order on the main actor; the assert assumes one.
 
-**Why:** `Ceiling::resumed()` resets `busy_since` and the state to Below on every resume, which was
-harmless while nothing held a box awake. With the #257 heartbeat it is not: a box whose ceiling
-prompt went unanswered sleeps with its job still running; the next connect wakes it (the status
-sweep connects to every remote workroom); the ceiling restarts; the box reads BUSY again; and the
-heartbeat holds it for another full ceiling. So "ask, then sleep" becomes "awake in 4 h chunks" on
-a paid box. Today no host that sleeps is wired in (containers never sleep), so nothing is exposed.
-
-Three more things the same reviews found, all moot until a host that sleeps is wired in:
-
-- **A base gets ask mode with nobody watching.** Every `AgentBootstrap.connect` sends
-  `ask && sleepsWhenIdle`, including provisioning's base connects, but only `RemoteHosts.connect`
-  starts a prompt watch. A long base job past the ceiling would sleep unasked. Send `ask: false`
-  from connects that start no watch.
-- **An idle window under 60 s defeats the heartbeat silently.** boxd's auto-suspend is
-  user-configurable; read the machine's timers at connect and warn below ~90 s.
-- **Prompt cards and stale verdicts.** Many hosts asking at once stack cards with no scroll, so one
-  can be pushed off-screen while its deadline runs; and a service that stalls after publishing IDLE
-  shows "idle", not unprotected (the badge's stall rule needs `busy`). On a sleeping host treat a
-  stalled reading as unknown, and bound the stack.
+**Depends on:** nothing.
 
-**How to start:** Keep the ceiling's awake time across a resume when the box was `Suppressed`, or
-make a resume after `Suppressed` stay suppressed until the user answers. Pin it with a
-`new_settings_apply_from_the_next_tick`-style table in `wakefulness/tests.rs`.
+**Priority:** P0 (chosen in /ship for #356, invocation 4, D6, 2026-10-07)
 
-**Depends on:** wiring `BoxdHostDriver` into `RemoteHosts` (#356). Land it with the
-`BoxdIntegrationTests` keep-awake case (P2, part of #356).
+## P1 — before GA
 
-**Priority:** P1, gating boxd in the app (chosen in /ship's adversarial review of #257, 2026-10-06)
+None open. The second-Mac discovery entry was dropped on 2026-10-07 (#348, closed as not needed).
 
 ## P2 — perf, correctness, and the next VCS phase
 
-### Put the keep-awake acceptance run in `BoxdIntegrationTests` (macapp) — part of #356
-
-**What:** Encode #257's live acceptance as a gated case in
-`macapp/WorkroomAppTests/BoxdIntegrationTests.swift`, so one command reruns it. Two machines:
-
-- **Control** (no agent): proves the job alone does not keep the box awake, which is what lets the
-  agent's result mean anything.
-- **Agent**: the claim under test.
-
-The hand run's third machine (ask mode, nobody answering) is dropped: "an unanswered prompt stops
-the heartbeat" is pinned by unit tests that run on every push, so the live case only re-proved
-boxd's timer.
-
-**Why:** It ran once by hand on 2026-10-06 and passed (results in "As built (#257)" in
-`docs/designs/remote-workrooms.md`). Nothing reruns it, so a boxd change to how it counts idle
-network time would quietly let busy boxes sleep. That costs nothing while boxd is not wired into the
-app, and costs users' work once it is. exe.dev (#259) needs the same proof.
-
-**How to start:** Two throwaway machines with 120 s suspend and hibernate timers
-(`--auto-suspend-timeout=120 --auto-hibernate-timeout=120`; `BoxdHostDriver.create()` takes no
-timers, so the case calls the CLI or the driver gains a test-only option). Run the same 6-minute
-job, with no network use, on each: the control with no agent, and the agent supervised with
-`--idle-timeout never`. Poll each machine's state every ~20 s from the Mac, and have the job log
-its ticks on the box. Pass: the control hibernates mid-job; the agent's machine logs every tick and
-hibernates within the idle window after the job ends. Destroy both, whatever the outcome (the
-derive tests' `remove(machine:)` pattern). About 15 minutes per run.
-
-**Depends on:** wiring `BoxdHostDriver` into `RemoteHosts` (#356). Runs only with
-`TEST_RUNNER_WR_BOXD_TESTS=1`, a `WR_AGENT_LINUX=1` build and the sandbox off, like the other
-cases there.
-
-**Priority:** build it with the boxd wiring (#356; eng review decision D10, 2026-10-06), and run it
-before any release that ships boxd. Moved out of P1 on 2026-10-06 because nothing that sleeps is
-wired in yet; originally deferred from plan `~/.gstack/projects/joelmoss-workroom/master-eng-review-20261005-233111.md`
-(T6), at /ship of #257.
-
 ### Give the keep-awake heartbeat its own thread (wr-agent) — #257 review
 
 **What:** Send the heartbeat from a thread of its own, fed the published verdict, instead of from

docs/designs/remote-workrooms.md

@@ -1369,8 +1369,12 @@ these are the subsystems that actually gate "a remote workroom is a real workroo
     instance. If undoing a step fails too, `Failure.rollbackIncomplete` names the host and grant
     still live, for the caller to record and finish, including a grant the enrolment itself
     could not cancel (`AgentEnrolment.GrantStillLive`); `destroy` reports the same way. The
-    driver's own failed cleanup is `HostDriverError.leftBehind`, naming what is still there, and
-    destroying a host twice succeeds, so a caller can retry what came after it. A failed grant cancel does not keep the box:
+    driver's own failed cleanup is `HostDriverError.leftBehind`, naming what is still there (and,
+    on boxd, the host it made), and destroying a host twice succeeds, so a caller can retry what
+    came after it. A create that leaves a machine running, from the driver's undo or a build's own
+    failed rollback, records it `failed`: a workroom's on its entry, a project's first base on the
+    project, which then refuses a second base until the project is deleted (#370). A boxd delete
+    also removes the derive snapshot named after the machine, which a failed derive can leave. A failed grant cancel does not keep the box:
     the key that would mint against the grant goes with it. An instance is never derived from,
     because its disk holds its key and credential helper. The workroom branches from
     `refs/remotes/origin/HEAD` after `git remote set-head origin --auto`, since `fetch` never
@@ -1892,6 +1896,71 @@ these are the subsystems that actually gate "a remote workroom is a real workroo
     its own. An IPv6-only box has no IPv4 default route and reports the error, and so does a default
     route with no gateway (`default dev wg0`), which a provider VM is not known to have. A changed
     setting reaches a remote agent on the app's next connect, not while connected.
+- **As built (#356, boxd in the app).** The app now makes, opens and deletes remote workrooms on
+  boxd, in Nightly and Dev builds behind `remoteWorkroomsPreview`, from New Workroom › Remote ›
+  boxd. Eng review 2026-10-06 (decisions D1-D11).
+  - **One driver table.** `RemoteHosts.DriverKey` is an enum over a container runtime (with its
+    Docker context) and a boxd org and account, and `RemoteHosts` holds any `HostTerminalDriver`.
+    Before, a boxd host had no key: a delete fell back to Docker, adopt dropped it, and a missing
+    container CLI read as its host being gone. A descriptor naming an unknown driver is refused,
+    never taken down with another. A boxd host records `driver: "boxd"`, its `org` and its
+    `account` (`boxd auth`'s `user_id`); the CLI keeps both as opaque fields.
+  - **Account, not only org.** Every personal account's org is nil, so after an account switch the
+    org check passed and a machine read as "not found" was taken for gone. The driver now checks
+    the account too before every create, derive, destroy and rollback.
+  - **Credentials.** Broker only (OQ20): a boxd workroom never borrows the Mac's `gh`, so the menu
+    asks for Codaset, and a create without it stops before anything is made. Whether boxd itself is
+    signed in is a CLI call, so the create checks it, naming `boxd auth login`.
+  - **Sleeping boxes.** An ssh login wakes a box, so a background read of a boxd host asks boxd
+    first (`machine get`, `standby` or `hibernated`) and leaves an asleep box alone; its row shows a
+    moon. Opening the workroom wakes it. boxd is asked at most once per host per 30 s about a box it
+    called asleep. When boxd cannot say (no CLI, a timeout, output the app does not expect), the
+    read leaves the box alone too and logs why, so a broken CLI cannot wake every box.
+  - **The ceiling across a wake.** A resume after an unanswered prompt carries the awake time from
+    before the sleep, so the agent asks again at once rather than granting a running job another
+    whole ceiling; the resume mask's IDLE ticks are ignored by the ceiling. Provisioning's connects
+    send the agent no settings (its `ask: false` could race a watched connection's `ask: true`), the
+    badge warns when a box's boxd idle timer is under 90 s, host prompt cards scroll past two, and a
+    stalled IDLE reading on a sleeping host shows "status unknown".
+  - **Not swept.** Unrecorded boxd machines are left alone: names carry no build, so one build's
+    sweep would take another's live workrooms (#284 has the boxd case).
+  - **Letting go of an idle box.** boxd's idle meter counts inbound traffic, and the first live run
+    (2026-10-06) found an attached box never slept: the service connection's ssh keepalives (15 s) and
+    the badge's polls (10 s) held it, idle or past an unanswered prompt. So a boxd host whose agent
+    reports IDLE (its published verdict, which an unanswered prompt also makes IDLE), whose workroom
+    is not selected, which forwards no port and has no pane attached is let go of: its connection
+    closed, no background read reconnecting it, its badge sending nothing, and its row showing "idle,
+    not connected" (an open moon) until boxd confirms it asleep. Selecting or opening the workroom, a
+    click (closing a pane, Keep awake), a pane attaching, or a port forward or BUSY reading that
+    arrives while the app is letting it go, takes it back. A remote pane's own ssh
+    keeps its box awake whatever the app does, so a host with a pane attached is never let go of; a
+    pane restored at launch attaches only once it is shown, and until then holds nothing. The app
+    polls each connected boxd host itself, so a row scrolled out of view still lets its box go; it
+    ignores IDLE readings for 10 s after a connect, so the read that connected the host is answered
+    first; and a host whose agent's wakefulness service has stopped is let go of too, as nothing on it
+    holds the box awake, unless its last reading was BUSY (the connection is then all that keeps the
+    job's box awake): that reading never changes and nothing will ask about it, so it holds the box
+    for one ceiling at most. A stalled service's reading, and a read that reached the agent and
+    failed, hold the box the same way: for one ceiling from the first such reading (a trusted reading
+    starts the count again), then it is let go. A read refused before it left, as for a host already
+    let go of, counts for nothing, and a connect starts a new count. The box's idle window is read on
+    every connect: a failed read keeps the last one, and an answer with no timer set clears it, so the
+    badge stops warning about a timer that was turned off. A host let go of is asked about again once
+    its boxd idle window (plus 30 s) has passed: if boxd says it is still awake, something on the box
+    holds it, so the app reconnects and hears its ceiling prompts.
+  - **Limit: selected workrooms and attached panes.** The selected workroom's host is never let
+    go of, nor is one with a pane attached, and a pane's own ssh holds its box awake regardless.
+    So a job past its ceiling, with its prompt left unanswered, keeps the box awake (and billed)
+    while its workroom is selected or any of its panes is attached. A pane counts as attached only
+    while its ssh lives: one whose ssh exited, or whose window closed, holds nothing, and a
+    reconnect attaches it again. The prompt's "let it sleep" takes effect only once that
+    workroom's panes are closed and another is selected.
+  - **Live runs (2026-10-06).** `BoxdIntegrationTests`, all nine cases, pass: the app's create and
+    delete (base in `/home/boxd`, records naming boxd with org and account, nothing left), #257's
+    keep-awake acceptance (the control slept mid-job, the agent's box logged every tick then
+    slept), and, after the change above, both app-attached checks: an idle box slept (the whole case,
+    machine creation included, took 318 s), and a box past a 60 s ceiling with its 30 s prompt unanswered slept mid-job. `machine
+    get --json` carries `status` and the timers as strings (`"auto_suspend": "120s"`).
 
 ## Phase 0 Results
 
@@ -2594,8 +2663,8 @@ disagreement passes every test on either side alone while presenting as an empty
    two open Macs is not built.
 
    *Not covered.* Local workrooms keep their layouts in `session.json` (TODOS, P3). A second Mac
-   still has to find the workroom and hold a key its host accepts (TODOS, P1), so the live
-   two-Mac test runs on the ssh fixture for now.
+   finding the workroom and holding a key its host accepts is not planned (#348, closed
+   2026-10-07), so the two-Mac test runs on the ssh fixture only.
 9. **What machine identity does a fork need re-minted?** **ANSWERED — Phase 0 item 6.** Measured duplicated across two live forks: `/etc/machine-id`, **`/proc/sys/kernel/random/boot_id`** (not on the list below, and it should be), both ssh host keys, and any on-disk secret. Re-minted by boxd: hostname and IP. Live processes carry across with their original pids. Not session UUIDs — those are client-minted
    and a template base has no sessions. The real list is ssh host key, machine-id, the agent's
    instance/socket identity, and any base-resident pty. Phase 0 item 6 measures which of these
@@ -2938,10 +3007,10 @@ disagreement passes every test on either side alone while presenting as an empty
   idle policy would otherwise have stopped it. That is the wakefulness service doing its job.
 - Start a long-running agent in it, quit Workroom entirely, reopen hours later, and land in the
   same session with the alternate-screen program correctly repainted.
-- Do the same from a **different Mac**. Open question 8 is answered and built (#255): the
-  layout and the sessions' workrooms live on the host, proven with two clients on the ssh fixture.
-  What still gates it on a real host is a second Mac finding the workroom and holding a key the
-  host accepts (TODOS, P1).
+- ~~Do the same from a **different Mac**.~~ **Removed (2026-10-07):** dropped from the gate
+  (#260), together with the second-Mac discovery work it needed (#348, closed as not needed). Open
+  question 8 is still answered and built (#255): the layout and the sessions' workrooms live on the
+  host, proven with two clients on the ssh fixture.
 - Close the laptop; the remote agent completes a `git fetch` and a `git push` with no client
   attached. (The concrete test that premise 6's revision was necessary.)
 - Changes, History, the diff viewer and the Files tree all work against a remote workroom and agree