Skip to content

Reduce workspace watch startup to four Git processes - #6135

Merged
ymichael merged 4 commits into
mainfrom
bb/cut-duplicate-git-processes-at-daemon-watch-star-thr_6g3uh44asm
Oct 22, 2026
Merged

ymichael merged 4 commits into
mainfrom
bb/cut-duplicate-git-processes-at-daemon-watch-star-thr_6g3uh44asm

Conversation

@ymichael

@ymichael ymichael commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

What was wrong

Workspace watch startup spawned eight Git processes per workspace. Provisioning, local state, and shared refs independently checked the same repository; provisioning also read the Git directory separately, and shared refs queried origin/HEAD separately from its ref listing.

What changed

Startup now uses four Git processes for a workspace with a resolvable origin/HEAD:

  1. One rev-parse --is-inside-work-tree --is-bare-repository --git-dir --git-common-dir supplies both provisioning checks. Local/shared reads reuse the known repository result.
  2. The existing ignored-directory status scan runs before watcher setup.
  3. The existing fresh local-state status scan runs after root watch readiness.
  4. One for-each-ref listing supplies branch refs and origin/HEAD's target using %(symref).

When the listing omits refs/remotes/origin/HEAD, the existing git symbolic-ref refs/remotes/origin/HEAD call remains as a fallback. Those workspaces use five processes, including repositories with no origin/HEAD or a dangling/unresolvable target. The same rule handles files and reftable storage, without reading ref-storage internals.

Both status scans remain because their ordering closes the setup window. WatchManager and host-watcher match main: no setup snapshot is cached. Snapshot fingerprints keep their existing representation, including the symbolic default-branch target, so retargeting origin/HEAD to another branch at the same commit still invalidates metadata. There are no daemon wire, CLI, or configuration changes.

How you verified

pnpm exec turbo run typecheck lint test --filter=@bb/host-daemon --filter=@bb/host-workspace --filter=@bb/host-watcher --concurrency=2 passed 13/13 tasks (four-checks-final.log).

  • Real-Git default-branch coverage runs against files and reftable storage: missing origin/HEAD, a resolvable target, a changed target at the same commit, dangling targets/chains, and local-branch fallback. Reftable cases skip only if Git cannot perform the migration. Both formats pass locally. Negative controls disabling either the unlisted-HEAD fallback or the listed symbolic target fail the assertions.
  • Provisioning coverage includes ordinary checkouts, linked worktrees, bare repositories, plain directories, and repository paths containing spaces/newlines where supported.
  • The setup-window regression still passes: after ignore discovery, it edits README and changes main→feature before root subscription activation, with no watch event delivered. It verifies content invalidation, dirty content status, and feature metadata. The same test failed on the former setup-cache implementation 1bfe2c22a0, reporting main. Before/after evidence remains in setup-window-before.log and setup-window-after.log.

Common-case startup measurements

Five fresh-process samples per version at 1, 5, and 20 workspaces, alternating main/branch and branch/main order. Baseline: origin/main c0013e2aba. Branch: a7c6c2c3f7. The source snapshots were captured at b859834f4e; the final head changes only test fixture initialization, and all measured production source files match it byte for byte (four-revisions.json). No older five/six-process samples are pooled into these results.

Used the existing external startup.mjs harness and startup-report.md method, with the same isolated clone and native-watched linked worktrees. The clock starts before replaceWatchSet, including provisioning, ignore discovery, and watcher setup. Watch-ready ends at the last workspace-root readiness callback; completion ends at the last Git child exit. The 1.6-second observation floor and 650-ms idle wait are excluded. Our package tests and fixture setup had finished before sampling; no browser, server, or status-request traffic was generated.

Workspaces origin/main Git processes Branch Git processes Reduction
1 8 4 50%
5 40 20 50%
20 160 80 50%

Counts were identical across all five samples in each cell. All 30 samples produced exactly one initial content invalidation and one metadata message per workspace, identical branch/default-branch/path/worktree values, and no raw watch events.

All workspace-root watches ready, median (min–max), ms:

Workspaces origin/main Branch
1 96 (91–132) 88 (84–96)
5 227 (222–307) 215 (203–247)
20 1630 (1508–7808) 2028 (1342–6408)

Startup completion, median (min–max), ms:

Workspaces origin/main Branch
1 175 (171–227) 156 (152–181)
5 398 (393–477) 383 (375–425)
20 1821 (1650–8460) 2104 (1438–6908)

The process reduction is consistent; timing is mixed. Readiness medians improved slightly at 1/5 workspaces and regressed at 20. Both versions have wide 20-workspace ranges. These samples do not establish a general latency improvement.

The original measure.mjs harness with zero event repetitions independently confirms 8→4, 40→20, and 160→80 processes. Separate one-workspace missing-HEAD and dangling-HEAD runs each confirm 8→5, with identical metadata between main and the branch (four-fallback-summary.json); these count checks are excluded from the timing tables.

Scripts, source snapshots, JSON traces, fixtures, logs, and the report remain outside the checkout under /Users/michael/.bb/thread-storage/thr_ci4wag9p6h/branch-perf/dedup/. The harness adaptations change import paths to preserved snapshots; instrumentation and settle rules are unchanged. four-run-samples.py reproduces the primary series, four-summarize.py validates counts/report parity and computes the tables, and four-fallback.py checks fallback counts. Primary traces are startup-four-*-<count>-<0..4>.json.

CI on final head a7c6c2c3f7 is green: 18 successful checks, 8 skipped. The PR remains open and unmerged.

AGENT GENERATED

@ymichael
ymichael marked this pull request as ready for review October 22, 2026 20:30
@ymichael ymichael changed the title Reduce duplicate Git processes during workspace watch startup Reuse repository detection during workspace watch startup Oct 22, 2026
@ymichael ymichael changed the title Reuse repository detection during workspace watch startup Reduce workspace watch startup to four Git processes Oct 22, 2026
@ymichael
ymichael merged commit b850353 into main Oct 22, 2026
26 checks passed
@ymichael
ymichael deleted the bb/cut-duplicate-git-processes-at-daemon-watch-star-thr_6g3uh44asm branch October 22, 2026 21:28
ymichael added a commit that referenced this pull request Oct 22, 2026
## Human comments

## What was wrong

GitHub pull request status was core. The daemon ran `gh`, the server
assembled and watched the status, and the app, SDK, CLI and Tasks and
GitHub plugins each reached it through core routes and commands. Core's
PR UI couldn't leave core because there was nothing outside core to read
from.

## What changed

**New bundled, default-enabled plugin: `plugins/github-pr-status`
(`bb--github-pr-status`)**
- `host.ts` / `host/git-host.ts`: the in-worktree lookup moved from
`packages/host-workspace/src/git-host.ts`. It covers the current branch,
upstream and fork remotes (`getPullRequestTarget`), the merge-queue
GraphQL query, and `gh pr ready|ready --undo|merge`. It runs on the
environment's own machine with the daemon's shell `PATH`.
- `server.ts`:
  - resolves the environment;
- serves a keyed `pullRequests` read, one row per environment id (up to
200), so a list of rows loads in one request;
- serves `markReady`, `markDraft` and `merge`, each of which re-checks
state before acting;
  - registers `bb pull-request show|ready|draft|merge <environment-id>`.
- `store.ts` is the single place reads are cached (10 s), re-checked
while watched, and published when they change.
- PR snapshots are kept by PR URL, so environments sharing a PR share
one snapshot. Persistence, cross-row reuse and webhooks can feed this
one place later.
- While a row is watched (`bb.rpc.experimental_whileWatched`), it
re-checks every 30 s when checks are pending, auto-merge is on, the PR
is queued or mergeability is unknown. That is the same rule as core's
`pull-request-watch.ts`.
  - Other triggers match main's:
    - `status-changed` and environment create/delete fetch fresh;
    - a completed turn re-checks through the 10-second cache;
    - `git-refs-changed` only marks the cached row stale;
- `metadata-changed` re-checks only when the environment's branch
actually changed (one environment read, no `gh`);
- in the app, a cached row is re-fetched on mount, and again when the
page is shown if it's older than 30 s (1 h once merged or closed). Main
got this from TanStack's `refetchOnMount: "always"` and focus refetch.
Here it uses only the public read hook's `refetch`, in
`pull-request-queries.ts`.
- It invalidates the row's tag only when the row changed, and never for
a result a viewer is already receiving.
- `contract.ts` is the public contract. The read returns the fully
assembled shape the UI renders: state, attention, checks, review,
mergeability, auto-merge and merge queue.

**Watched reads**
- `bb.rpc.experimental_whileWatched` comes from `bb/5939-recut`,
mechanism only: the `plugin-read` subscription target, the server
registry and producer lifecycle, app query hooks holding the
subscription, per-target subscribe validation, and fake-host support.
- Left out: the SDK `experimental_watchRpc` /
`experimental_watchDiscovery`, the CLI `--watch` flags, and
plugin-signal scoping.

**Core reads and acts only through the plugin's public contract**
- The composer banner segment, Info row and thread view use
`experimental_useRpcKeyedQuery` / `experimental_useRpcMutation` with
`pluginId: "bb--github-pr-status"`
(`apps/app/src/hooks/queries/pull-request-queries.ts`). These are the
same calls another plugin makes, so moving the UI later is a cut and
paste.
- The app's RPC hook host now allows those hooks outside a plugin slot
when `pluginId` is explicit.
- `bb thread show`, Tasks (`listTaskPullRequests`, now one batched read)
and the GitHub plugin's `pullForThread` call the same RPC.

**Deleted from core**
- The server PR watcher, route, cache, `environment-pull-request` tag
and subscription target, and the PR action branches.
- The SDK `environments.pullRequest` / `markPullRequestReady` /
`markPullRequestDraft` / `mergePullRequest`, and `bb environment
pull-request …`.
- The `ThreadPullRequest` / `GitHostPullRequest` domain schemas.
- The daemon's `workspace.pull_request` and
`workspace.pull_request_action`. **`HOST_DAEMON_PROTOCOL_VERSION` 250 →
251**, compatible `[250, 251]`. 250 stays compatible because the server
no longer sends either command. 246–249 stay removed, as #6018 intended.

**`experimental_useSidebarThreadPullRequest`**
- Removed from `@get-bb/plugin-sdk/app`, the Plugin Guide and the docs.
- Published bundles keep working: the app installs
`retiredPluginSdkAppMembers` on the plugin runtime, backed by the
plugin's read. It returns `{ isLoading: false, pullRequest: null }` when
the plugin is disabled.
- **Builds never fail on it.** bb builds git-sourced marketplace plugins
at install and update, and rebuilds path and builtin plugins when the
SDK changes. So the `bb-plugin-build` runtime shim keeps the name in its
`@get-bb/plugin-sdk/app` export list. Importing it is a build warning
with the replacement call, which `bb plugin build`/`dev` print and the
server writes to the plugin's log.
- Migration guide:
`bb-plugin-authoring/references/frontend-pull-requests.md`. The runtime
shim is recorded in `docs/api_to_audit.md`.
- Plugin SDK 0.6.72.

**No wake.** Plugin host calls never wake a suspended host. A read on an
asleep or offline machine returns `unavailable`, which the UI renders as
no PR. Ready, Draft and Merge fail with "<machine> isn't available …".
**Behaviour change:** reads no longer wake a suspended host. On main,
`workspace.pull_request` used the `work` wake policy, so opening a
thread whose machine was suspended woke it.

## How you verified

**Request measurement in the real app.** Main and this branch ran as two
`pnpm start:worktree` instances side by side. Each had a thread on a
project checkout of PR #6117's branch while its checks were pending
(`checks_pending`, 0 failed). In each, a headless page opened the thread
and every `/api` request was recorded for 3 minutes, both runs at the
same time:

| | PR requests | when (s) | all `/api` requests |
|---|---|---|---|
| main | 6 × `GET /environments/:id/pull-request` | 1, 32, 64, 95, 127,
158 | 84 |
| branch | 6 × `POST /plugins/bb--github-pr-status/rpc/pullRequests` |
1, 33, 64, 96, 127, 159 | 88 |

The other `/api` differences changed from run to run (hosts, server-move
and execution-options in this run; system config, providers and timeline
in another). They come from background activity in the freshly restarted
branch instance and its extra sidebar threads, not PR traffic. An
earlier run found two bugs that are fixed here with tests:
- a re-check right after open published a result the opening read was
already fetching, costing one extra request;
- a watch that went idle when checks finished never resumed when a new
CI run started.

**Busy thread.** Main and this branch ran side by side again, each with
a thread on its own scratch PR in a private test repo
(`ymichael/test-project` #303 and #304, both closed and their branches
deleted afterwards). Each thread was open in a page. Both received the
same agent turn at the same moment: three commits, a `git fetch`, a
third commit between them, and a `git push`, which also started the
repo's CI. Over 5 minutes, a process monitor counted every `gh` started
by each instance, attributed by process ancestry and confirmed by
working directory. The page counted PR read requests. Two runs gave
identical counts:

| | `gh` calls | PR read requests |
|---|---|---|
| main | 7 (4 `pr view`, 3 merge-queue `api graphql`) | 4 |
| branch | 5 (3 `pr view`, 2 merge-queue `api graphql`) | 2 |

Before this fix the branch also re-checked with `gh` on every
`git-refs-changed` and `metadata-changed`, so a thread that commits and
pushes would have exceeded main.

**Focus refresh.** Main (`f909788625`) and this branch ran side by side
with the same open PR. A page on each hid and showed itself 5 times over
4 minutes, with gaps from 20 s to 60 s, so some shows were under the 30
s stale time and some over. A process monitor counted `gh` per instance:

| | `gh` calls | PR read requests |
|---|---|---|
| main | 5 | 5 |
| branch | 5 | 6 |

The extra read is the realtime connection resuming at the same moment
the page showed. Both reads hit the plugin's 10 s cache, so there's no
extra `gh`.

**Published plugins that use the retired hook.** On the review app, I
installed four git-sourced Community marketplace plugins that call
`experimental_useSidebarThreadPullRequest`. Each went through `bb plugin
install <plugin>@bb-community`, so bb cloned and built it at its
published commit. Each was then used as the sidebar thread list:

| Plugin | Commit | Builds | Warning in `plugin.log` | Renders PR badges
|
|---|---|---|---|---|
| t3sidebar | `8b535729dd` | ✅ | ✅ `src/ThreadCard.tsx:6` | ✅ |
| dockside | `e8736f3858` | ✅ | ✅ `components/inbox/thread-card.tsx:7` |
✅ |
| radar-sidebar | `9a20bfab7e` | ✅ | ✅
`components/radar/RadarThreadRow.tsx:23` | ✅ |
| thread-inbox | `c0981fb6fe` | ✅ | ✅ `src/ThreadCard.tsx:18` | ✅ |

gtd-sidebar fails to install on main too ("Could not resolve
@get-bb/plugin-sdk"), so it's unrelated. This test caught a real bug: T3
Sidebar crashed with a React hook-order error. React Compiler only
treats `use…` names as hooks, so it cached the
`experimental_useRpcKeyedQuery` call inside `useEnvironmentPullRequest`
and skipped it on re-renders with unchanged arguments. Core's own
callers pass a fresh options object on every render, which hid it. The
hooks are now imported under `use`-prefixed local names, and all four
plugins render.

**UI parity.** Screenshots of the same thread on main and on the branch
show the same composer banner segment (PR #6117 with the Merge split
button) and the same Info row (#6117, Open, Checks running). `bb thread
show` printed the PR through the plugin. On this PR, `bb pull-request
draft <env>` converted it to draft. The banner then showed "PR #6125 ·
Draft" with Mark ready, and clicking Mark ready showed the loading
toast, made the PR ready on GitHub, and returned the banner to the open
state with Merge.

**Tests**
- New plugin tests (75):
- host-entry fork, upstream and detached cases using a fake `gh`, moved
from host-workspace;
  - the parse, action-argument and merge-queue table;
  - the assembly table, moved from the server;
  - store polling, publish-on-change and the viewer race;
  - server reads, refusals, the no-wake refusal and the CLI.
- Also added: the `bb-plugin-build` retired-import warning test, the
focus-refresh staleness table and refetch tests, Tasks batched reads and
the disabled-plugin case, CLI `thread show` through RPC, and a hub test
for the `plugin-read` target.
- After merging #6135: `pnpm exec turbo run typecheck lint
--filter='...[origin/main]'` passes 162/162, and `turbo run test
--filter='...[origin/main]'` passes 115/115. That includes #6135's
host-workspace default-branch tests and the host-daemon
`watch-manager-startup` setup-window test. The host-workspace merge kept
#6135's code. This branch only removes the PR lookup from it.

**Not covered**
- `gh` ran only on a local host. Remote hosts use the same plugin
host-artifact path as the other host plugins but weren't exercised.
- Merge wasn't run against a real PR. It uses the same `gh pr merge
--<method>` arguments, covered by the host tests.

> AGENT GENERATED

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant