perf: one list read per refresh, parallel loads, no duplicate watcher refresh (v0.12.0)

AGENTS.md

@@ -21,6 +21,11 @@ src/
                         (queueGet) and mutations (callControlApi). Extracted from
                         server.ts so the auth and transport guards are
                         unit-testable without booting the server.
+  shared-read.ts        One in-flight upstream read shared by concurrent callers, and
+                        the invalidate-after-write wrapper every mutation goes through.
+  refresh-rules.ts      The UI's ordering rules as pure functions: last-started-wins,
+                        and when a watcher event is already covered by a refresh.
+                        Also owns WATCH_DEBOUNCE_MS, imported by server.ts and index.ts.
   queue-token.ts        Loads the plugin's client token from the fixed file
                         under $HOME, failing closed on mode or content.
   ws-guard.ts           The WebSocket upgrade decision, as a pure function of
@@ -73,6 +78,11 @@ importable. esbuild and `tsc` (`allowImportingTsExtensions`) both accept `.ts`.
 ## Invariants
 
 - **The plugin never reads or writes queue YAML directly.** Every mutation goes through `control-api.ts` to the MCP control API, inheriting its transition validation, `fcntl` locking, and atomic writes. Since v0.11.0 every read does too (`queueGet` → `GET /tasks`, `GET /tasks/{id}`), inheriting the queue's TTL, dead-letter and status rules; before that this file globbed the YAML and applied none of them. A new mutation means a new control-API action, never an `fs.writeFile`; a new read means an API call, never a `yamlLoad` of a queue file. The `fs.watch` on the queue root is a change trigger and reads nothing.
+- **Reads in parallel, mutations never.** A refresh issues its three reads concurrently, and a button's refresh reads the list and the detail concurrently. Each button's mutation is a single awaited call, and `pendingActions` in `index.ts` drops a second click on the same task (Start included) before any `confirm()`. task-queue-mcp's audit accepted an unpark race (part 1, F-01) on the premise that clients do not send duplicate mutations. Do not relax that guard, and do not fire a mutation alongside anything else for the same task.
+- **The unfiltered list is shared as a READ, never as a cached RESULT.** `/tasks` (no filters) and `queueIndexByPrefix` join one in-flight `GET /tasks` through `SharedRead`. Nothing is reused once that read settles, so there is no time window. Every mutation goes through `mutate()` → `invalidatingWrite`, which invalidates before the route responds, whatever the outcome. The watcher invalidates on the raw fs event. Tests pin both, and pin that `server.ts` calls `callControlApi` only there. A new mutation path that calls `callControlApi` directly would serve a pre-mutation list after its own click. Filtered lists and the dead-letters read are separate queries and stay unshared.
+- **Refresh results apply last-started-wins.** Parallel loads can resolve out of order, so `loadTasks` and `loadTaskDetail` apply a result only if no newer load has begun (`Latest`). The detail load also drops a result for a task no longer selected.
+- **A watcher event already covered by a refresh is skipped.** `coveredByRefresh`: if a refresh whose list read **succeeded** started at or after `eventAt - WATCH_DEBOUNCE_MS` (the latest the reported change can have happened), it read after the write. The check runs when the 2 s UI debounce fires, not when the event arrives, so a refresh still in flight at arrival has settled by then. A failed refresh covers nothing. That removes the second full refresh a second after every button press. Changes made elsewhere still refresh. Lateness only errs toward refreshing. If you change the backend debounce, change the constant, which both sides import.
+- **Parallel requests do not make the API parallel.** task-queue-mcp serves HTTP reads on worker threads, but the parse is CPU-bound Python, so two concurrent list reads take as long as two sequential ones (measured 2026-09-30: 0.215 s sequential vs 0.229 s parallel). The saving is in making fewer reads, not in overlapping them.
 - **`truncated` is rendered, never dropped.** `GET /tasks` returns at most 1000 records and says when it cut some off. The header and the dead-letters badge show it. If a view ever needs more than one page, report the number and the use rather than paging silently or asking for the cap to be raised.
 - **`ControlAction` must match the MCP's route set.** The union type in `control-api.ts`, the route regex in `server.ts`, and the MCP's custom routes are three copies of one contract. Change one, change all three. The two copies that live in *this* repo are now pinned to each other by a source-level test in `control-api.test.ts` — nothing detected the drift before, because adding an action to the union alone compiles and the failure mode is a button that 404s against the plugin's own backend. The third copy is in another repo and still needs a human.
 - **The task-queue vocabulary lives in `vocabulary.ts`, once, and is gated against its owner.** This plugin does not own the queue's statuses, task types, or workflow modes — `task-queue-mcp`'s `src/tools/queue.py` does. It used to carry four partial hand-written copies (`STATUS_ORDER`, `NON_TERMINAL_STATUSES`, `DETAIL_NON_TERMINAL_STATUSES`, the `statusColor` switch); none of them learned about `routing-failed`, so for months the status most in need of an operator sorted *below* `cancelled`, rendered the same grey as `parked`, and was not offered by the status filter. `manual-then-auto` was the same omission one field over. Two mechanisms hold the line and they are different in kind: `npm run gate:vocabulary` fetches the MCP's `main` and fails on any difference, which catches "upstream changed and we did not"; and the UI maps are `Record<Status, …>` keyed by the vocabulary itself, so adding a status without giving it a sort position and a colour is a `tsc` error rather than a silent fallthrough to `?? 9` and `muted`. Do not weaken either — a `Record<string, …>` accepts anything and covers nothing, which is precisely how this happened.
@@ -114,7 +124,7 @@ npm test
 npm run gate:vocabulary  # parity with task-queue-mcp main — needs network, by design
 ```
 
-Tests cover `queue-token.ts` (real files in a tmpdir: missing, empty, directory, every group/other mode bit refused, `0600` and `0400` accepted, and no error carrying the file's content), `control-api.ts` (the token gate, the `X-Task-Queue-Token` header and the absence of `Authorization` and the retired secret header, the read helper and its query builder, the manifest's `env:` grants, task-id validation, header and body shape per action, transport-failure mapping, pass-through of the MCP's authorization rejections, and the union/route-regex drift gate), `dead-letters.ts` (the real dispatcher record shape including the `Date`-valued `created` js-yaml hands back, the id-less and reason-less fallbacks, and the grouping rule against the live seventeen-identical-reasons case), `ws-guard.ts` (all three upgrade cases, including the loopback-with-no-Origin one that v0.4.0 broke), `launch-policy.ts` (every closed-set rejection, whole-document rejection, and both argv shapes), `path-guard.ts` (a **real** symlink escape, traversal, the `/comms-other` sibling case — with real files in a tmpdir, because a mocked `fs` cannot demonstrate that realpath runs first), `launch-log.ts` (round-trip against the real `launchLogName`, the live bare-UUID orphans, path-shaped route ids, fence extraction including the dropped unterminated case, and the birthtime-after-mtime fallback), `vocabulary.ts` (every status has a sort position and a colour, `routing-failed` sorts above `in-progress` and is not muted, the derived non-terminal set, and the `manual-then-auto` pass-through through `buildLaunchArgv`), `gates/python-sets.ts` (the real `queue.py` shape including interleaved comments, and the two constructs that must NOT parse as string sets — a derived set and an annotated dict), and the reconnect schedule. The UI panels are not otherwise unit-tested — verify them in CloudCLI after `./deploy.sh && pm2 restart cloudcli`.
+Tests cover `queue-token.ts` (real files in a tmpdir: missing, empty, directory, every group/other mode bit refused, `0600` and `0400` accepted, and no error carrying the file's content), `control-api.ts` (the token gate, the `X-Task-Queue-Token` header and the absence of `Authorization` and the retired secret header, the read helper and its query builder, the manifest's `env:` grants, task-id validation, header and body shape per action, transport-failure mapping, pass-through of the MCP's authorization rejections, and the union/route-regex drift gate), `dead-letters.ts` (the real dispatcher record shape including the `Date`-valued `created` js-yaml hands back, the id-less and reason-less fallbacks, and the grouping rule against the live seventeen-identical-reasons case), `ws-guard.ts` (all three upgrade cases, including the loopback-with-no-Origin one that v0.4.0 broke), `launch-policy.ts` (every closed-set rejection, whole-document rejection, and both argv shapes), `path-guard.ts` (a **real** symlink escape, traversal, the `/comms-other` sibling case — with real files in a tmpdir, because a mocked `fs` cannot demonstrate that realpath runs first), `launch-log.ts` (round-trip against the real `launchLogName`, the live bare-UUID orphans, path-shaped route ids, fence extraction including the dropped unterminated case, and the birthtime-after-mtime fallback), `vocabulary.ts` (every status has a sort position and a colour, `routing-failed` sorts above `in-progress` and is not muted, the derived non-terminal set, and the `manual-then-auto` pass-through through `buildLaunchArgv`), `gates/python-sets.ts` (the real `queue.py` shape including interleaved comments, and the two constructs that must NOT parse as string sets — a derived set and an annotated dict), `shared-read.ts` (one upstream GET for concurrent callers against an injected fetch, no reuse after settle, a post-mutation list that is fresh while a pre-mutation read is still in flight, invalidation on refused and failed writes, and failures never cached), source pins that `server.ts` routes every mutation through `mutate()` and invalidates in the watcher, `refresh-rules.ts` (a stale out-of-order response losing to a newer one, and the watcher-skip rule's own-action, elsewhere, mid-refresh and late-delivery cases), and the reconnect schedule. The UI panels are not otherwise unit-tested — verify them in CloudCLI after `./deploy.sh && pm2 restart cloudcli`.
 
 Two build/test gotchas worth knowing before you touch either script:
 

CHANGELOG.md

@@ -2,6 +2,48 @@
 
 All notable changes to this project will be documented in this file.
 
+## [0.12.0] - 2026-09-30
+
+Fewer, parallel reads per refresh. Programme `task-queue-read-perf-2026-09` part 3;
+vikunja#1003. **Includes 0.11.1's security fix** (redirects never followed on
+token-bearing requests), so deploying 0.12.0 supersedes the pending 0.11.1 deploy.
+No change to authentication, the token file or the manifest's grants.
+
+### Changed
+
+- **One upstream `GET /tasks` per refresh, down from two.** `/tasks` (unfiltered) and
+  `/headless-runs` used to make the same list read each. They now join one in-flight read
+  (`shared-read.ts`). Only the read is shared: nothing is served once it settles. Every
+  mutation route and a Start's history write invalidate it before responding, and so
+  does the queue watcher, so a list requested after a mutation is always read after it.
+  Filtered lists and the dead-letters read are not shared.
+- **The tab's three reads run in parallel**, and after a button press the list and the
+  task detail are re-read in parallel. A failed runs or dead-letters read still never
+  blanks the task list. Results apply last-started-wins, so a slow older refresh cannot
+  overwrite a newer one.
+- **No second refresh after a button press.** The watcher's `tasks` event for the tab's
+  own write arrives about a second later and used to trigger another full refresh. It is
+  now skipped when a refresh already started after the change it reports. Changes from
+  agents and the dispatcher still refresh live.
+- **A second click on the same task is ignored while its action is on the wire**, Start
+  included (a double click used to be able to launch two sessions). Mutations are never
+  sent in parallel or twice. task-queue-mcp's accepted unpark race (F-01) depends on this.
+
+### Measured
+
+Built backend run locally against forge's live task-queue-mcp v0.13.0, via a counting
+proxy (park stubbed at the proxy, so the queue was not written). Median of 7:
+
+| | 0.11.1 | 0.12.0 |
+|---|---|---|
+| Tab load | 0.372 s, 3 upstream reads | 0.250 s, 2 upstream reads |
+| Park + refresh | 0.382 s, 4 upstream reads | 0.268 s, 3 upstream reads |
+| Watcher refresh after that Park | 3 more reads | skipped (by rule and unit tests; seen in CloudCLI only after deploy) |
+
+The saving comes from the dropped read, not from overlap. task-queue-mcp's list parse is
+CPU-bound, so two concurrent reads take as long as two sequential ones (0.229 s vs
+0.215 s).
+
 ## [0.11.1] - 2026-09-30
 
 Security fix from the `operator-panel-2026-09-p2-queue-read-api` audit (F-01, Medium;

README.md

@@ -32,7 +32,7 @@ Routing writes through `task-queue-mcp` means mutations inherit its transition v
 - **UI** (`dist/index.js`) — renders the tab panel: a filterable task list and a detail view with history timeline, amendments, and context-ref previews.
 - **Backend** (`dist/server.js`) — HTTP + WebSocket server launched by CloudCLI. Picks a free ephemeral port at startup and reports it to CloudCLI as JSON on stdout. The UI reaches it through CloudCLI's plugin RPC API (`api.rpc()`).
 
-Live updates arrive over WebSocket: the backend watches the queue directory and pushes a `tasks` event when files change; the UI debounces, then re-reads through the API.
+Live updates arrive over WebSocket: the backend watches the queue directory and pushes a `tasks` event when files change; the UI debounces, then re-reads through the API. A refresh makes its three reads (tasks, headless runs, dead letters) in parallel, and the backend answers the first two from one upstream `GET /tasks`. After a button press the UI refreshes once, and skips the watcher's event for the same write (since v0.12.0).
 
 If the API truncated a read (it returns at most 1000 records per call), the header says **truncated: showing N of M** and the dead-letters badge says the count may be low. It is never hidden.
 

manifest.json

@@ -1,7 +1,7 @@
 {
   "name": "task-queue",
   "displayName": "Task Queue",
-  "version": "0.11.1",
+  "version": "0.12.0",
   "description": "Task queue dashboard \u2014 view, filter, and launch agent tasks.",
   "author": "TadMSTR",
   "icon": "icon.svg",

package.json

@@ -1,6 +1,6 @@
 {
   "name": "cloudcli-plugin-task-queue",
-  "version": "0.11.1",
+  "version": "0.12.0",
   "private": true,
   "type": "module",
   "scripts": {