Description
Problem
Main now has part of the "report goes through publish_handoff" change, and the outros point at files that do not exist.
#1141 (merged 2026-08-27) added this clause to the publish_handoff description: "do not write the report to a file yourself". It is at src/agent/tools/handoff.ts:30-33 on main (d8486dc), and the pi tool uses the same text. The rest of the code still asks for the file or says it exists:
skillPromptstill says "write a brief markdown report to ./${reportFile}" (src/agent/agent-prompt.ts:54). Every program with askillIdgets this line.- The context-mill skills still say to write it.
context/skills/data-warehouse-source/description.md:99says "write the report file requested by the wizard".context/skills/audit/references/6-report.md:7and:52say toWriteposthog-audit-report.mdto disk. context-mill#306 moves these skills topublish_handoff. It is still a draft. - The outros show the file name and never check the file exists.
src/agent/runner/sequence/linear.ts:294-300passesrun.reportFilethrough.src/ui/tui/screens/OutroScreen.tsx:87-93prints "Check ./… for details", andsrc/ui/tui/exit-line.ts:91-92adds the same line on exit.src/ui/tui/screens/audit/AuditOutroScreen.tsx:64-76prints "Report saved to:". The audit success message atsrc/lib/programs/audit/index.ts:48, the ledger row label atsrc/lib/programs/audit/seed.ts:99-103and the "Write report" slide atsrc/ui/tui/screens/audit/slides/writeReport.tsx:25also name the file. - On main,
publish_handoffwrites a file only when the host setsPOSTHOG_HANDOFF_OUTPUT_PATH. The TUI does not set it, so nothing is written to the project. - The wizard-workbench warehouse grader also expects the file. Its "report written" check passes only when
result.reportFile.existsis true (PostHog/wizard-workbenchservices/wizard-ci/warehouse-checks.ts:813-820), so a run that follows the tool description fails it.
Evidence
Each run follows one instruction or the other. We saw this in a real-TUI sweep of all nine programs on the refactor stack head d938a43 (#1307), which has the same code for this path as main:
- warehouse (
revenue/stripe/stripe-saas-demo): the agent called onlypublish_handoffand marked "Write setup report" completed. The outro says "Check ./posthog-warehouse-report.md for details". No file was written. - audit (
audit/posthog-demo-3000): the agent resolvedwrite-reportas pass: "Final markdown report delivered through the wizard handoff." The outro says "Report saved to: /posthog-audit-report.md". No file was written. The Release A top (200961f) did the same. - metrics (
express-todo, orchestrator flow): the report went only to a notebook. The orchestrator outro does not name a file, so it shows nothing false. - ai-observability wrote its file and also called
publish_handoff. On 200961f, warehouse wrote its file too. The result depends on which instruction the model follows.
The stack carries the same lines at other positions (src/agent/agent-prompt.ts:54, src/agent/tools/handoff.ts:33, src/agent/runner/sequence/linear.ts:314, src/ui/tui/screens/OutroScreen.tsx:107, src/ui/tui/screens/audit/AuditOutroScreen.tsx:67, src/programs/resolve-run-definition.ts:56), so this is not a stack regression.
Prior work
#1047 (open draft, last updated 2026-07-31, conflicting) makes publish_handoff the only report channel. It changes skillPrompt, stops programs naming a .md file, and shows the file in the outro only when one was written. context-mill#306 is its skill-side pair. Both are stale, and #1047 predates #1141, so it does not describe the contradiction main has now. Its base is ce8b70d, and the files have moved since: src/lib/agent is now src/agent, and src/lib/wizard-tools is now src/agent/tools.
#811 (open draft, stale since 2026-07-06) gates the linear-runner outro on the file existing. It is on old paths and does not cover the audit success message or screen.
Suggested fix
- Now: pick one instruction. Either drop "do not write the report to a file yourself" from
PUBLISH_HANDOFF_DESCRIPTIONuntil the rest moves, or changeskillPromptto order apublish_handoffcall and stop naming the file. - Until one channel remains, show a report path only when that file exists in
installDir. fix(outro): only promise the setup report when it exists on disk #811 did this for the linear runner. The same check needs to cover the audit outro screen, the audit success message and the exit line. - Longer term: land feat(wizard-tools): publish_handoff creates the notebook, outro opens it #1047 and context-mill#306 together, or close them and do the same change on the current paths. Update the workbench "report written" check to accept the published handoff when the program no longer names a file.
Activity