Repository navigation
M2 completion: make a run answer (record of work already landed) #341
Description
Part of #325. Opened as a record rather than as a work item: the work is done, and this exists so the epic does not read as though it never needed doing.
Why this issue exists
#329 closed on 2026-08-12 with every task in its scope complete — durable runs, an authenticated HTTP surface, a gated rail, 100% coverage, all six gates green. Its acceptance criteria never said a run answers the question it was asked, so nothing checked whether one did.
Nothing could check, either: until #336 the rail could not start a run on any connection a zero-config deployment ships (backlog B22), so the surface had never been driven end to end against a real model. The gap was invisible while the door was locked.
The three PRs below closed it. They reference no issue, which is what prompted this record.
The evidence
Nine agent runs against Gemini and the bundled SQLite and LibreDB samples, read back from the durable ledger (.workflow-data/streams/chunks/*.bin) rather than from the screen. Two of them were the maintainer's own.
| Started | Run | Mode | Status | Tools | Events | Report |
|---|---|---|---|---|---|---|
| 13:29:02 | arun_23daf33 | agent | failed | 3 | 7 | none |
| 13:29:52 | arun_7529b42 | agent | failed | 11 | 37 | none |
| 13:41:11 | arun_0cbe5fc | planning | succeeded | 0 | 3 | none |
| 13:42:01 | arun_59aa818 | agent | succeeded | 5 | 17 | none |
| 13:48:29 | arun_5b75c31 | planning | succeeded | 0 | 3 | none |
| 13:49:16 | arun_eceb045 | agent | unfinished | 5 | 16 | none |
| 13:52:05 | arun_5b5fb3a | planning | failed | 0 | 3 | none |
| 13:52:28 | arun_2868d80 | agent | failed | 0 | 4 | none |
| 13:53:53 | arun_bb6cfd4 | planning | failed | 0 | 3 | none |
Nine runs, zero reports. A planning run's entire ledger was three events, which is the shape of a run that had nothing to say.
Every layer M2 built did its job — schema capture, drafting, policy-bound read-only execution, EXPLAIN, artifact storage, the ledger, the live timeline. Nothing made a run answer.
Findings
F1 — Planning mode was mute by construction. The default mode called the model, received a plan and discarded it. The only prose-carrying event (report-composed) requires evidence; evidence requires an artifact or a context snapshot; planning has an empty tool set by contract. The plan had nowhere to go. Fixed in #338 by closing-statement, an event whose whole point is that it cites nothing.
F2 — A run could end succeeded without answering. compose_report is a tool the model may simply not call, and nothing checked that it did. One run inspected the schema, ran the average-salary query, ran EXPLAIN on it and read the indexes on salary — a competent investigation of exactly the question asked — then finished succeeded with the answer nowhere. #338 records stopReason, so the rail now states "the model stopped without composing a cited report" and shows what the model did say. The status word itself is deliberately unchanged — see the hand-off below.
F3 — The default connection could not run the agent, and the rail did not say so. Phase 1 targets PostgreSQL and SQLite; the LibreDB sample has no database-native read-only execution profile, and it was the connection a zero-config deployment landed on. The run was accepted, spent a model call and failed as internal. Fixed twice in #338: an engine-unsupported reason with words a user can act on, and the SQLite sample now leads the built-in seeds so a first run starts where the agent works.
F4 — One model call could spend the entire run budget. Originally recorded as a "398-second hang"; measuring from the ledger instead of from polling shows the run ended at 300.0 s — exactly AGENT_RUN_DEADLINE_MS. The deadline worked perfectly, and the 398 was an observation time reported as a duration. What was real: the run deadline was the only bound on a single call. #339 adds AGENT_MODEL_TURN_TIMEOUT_MS (90 s) and reports model-timeout separately from deadline-exceeded. Why that one call never answered is still unknown — it now costs 90 s rather than the run.
F5 — Turn-budget exhaustion lost the work it had done. AGENT_MAX_MODEL_TURNS is 16. One run obtained all eight table counts in a single query, then re-verified each table individually and hit the ceiling, ending failed with no partial statement. #338's stopReason: turn-limit now says so and points at what was gathered. The model's strategy is what wastes turns, and that belongs to #330's goal verifiers.
F6 — A log line that named neither the connection nor the reason. logger.error(message, error?, context?) takes the error second; four call sites passed their context object there, so the formatter rendered Unknown: [object Object] and the context slot stayed empty. The loudest fired on every managed-connections request. Fixed in #339, with a scanning test guarding the class because error?: unknown makes the shape uncatchable by the compiler.
What landed
- fix(agent): let a default deployment start a run on its seed connections #336 —
resolveAgentRunConnectionId: a run may start on a managed connection, or on the editable copy of a seed while it still matches the descriptor the browser fetched. Review then found the other half: the connection editor silently droppedseedId/managed/group/agentUser/agentPasswordand inventedhost: "localhost"for file-addressed engines, so a rename never survived a reload — user-facing, and unrelated to the agent. - feat(agent): let a run say what it concluded, and how it stopped #338 —
closing-statementandstopReasonon the ledger;AgentRunFailureReasonsplit from three to six (model-rate-limited,model-unauthorized,engine-unsupported); the SQLite sample leads the seeds. - fix(agent): bound one model call, and make four log lines say what happened #339 — the per-call ceiling; the four log call sites and their guard; the B15 retention note moved out of the report section, because only a run that composed a report ever explained why its "Show result" controls had gone.
Handed to #330, not deferred
B24 — a run that answers nothing is still succeeded. AgentRunTerminalStatus has no word for "finished, and did not answer", and #330's acceptance criteria already name needs_input, a status this codebase does not have, plus a policy gate requiring "citation present for every final finding".
failed was deliberately not spent on it: it would be wrong for planning mode, which has no tools and cannot produce evidence, so a planning run that returned a good plan did exactly what the mode is for; changing ledger semantics twice means every past run is re-read under a new meaning twice; and #330 asks for goal verifiers per template, which implies the rule differs by workflow type rather than being one predicate over every run.
Also open, tracked in docs/BACKLOG.md
- B23 — seed eligibility is decided against the browser's last descriptor fetch, so a seed repointed server-side mid-session is not seen until the next fetch.
- D3 —
POST /api/db/test-connectionopens its own provider, so testing a connection to an already-open single-writer file fails on the first handle's exclusive lock. The connection modal tests before it saves, which makes the built-in LibreDB sample uneditable.
The process lesson
M2 passed on unit tests, 100% line coverage and a rail that renders. None of those can observe that a run said nothing, because the code ran correctly — what was missing was a requirement. The existing test asserted the model's prose was returned to the caller, which is a correct test of the wrong thing: a user reads the ledger, not a return value.
That is the argument for #330's eval gates being built before its templates, and it is recorded in that issue.
Activity
cevheri commentedon Aug 12, 2026
Closed on creation: this is a record, not a work item. All three PRs are merged (#336
25f5b4f, #338793cae7, #339f54824c), and what was deliberately not done is tracked as B24 against #330.Verified on merged main with a cleared browser: a first run lands on the SQLite sample, and an agent run reports
Run failed — "The run reached its step limit before it finished. What it had gathered is above."Before this work the same run showed a bareRun failedwith no reason and stored results with no explanation.feat(agent): measure whether a run answered, before building anything…
feat(agent): give a run a second axis — what it is FOR, decided once …
fix(agent): remind a run that read the database and then narrated