refactor: split the tools out of programs by gewenyu99 · Pull Request #1384 · PostHog/wizard

gewenyu99 · 2026-09-29T09:28:16Z

Phase 2 of Release C (#1320), PR 8 of 16, after #1383. Review PR. PRs 1 to 14 of this phase split one change into reviewable pieces and compile only as a stack: review them one by one, then land the whole phase together.

Problem

On main, mcp add, mcp remove, the MCP tutorial, Slack and doctor are registered as programs, though none of them runs runProgram. skill list, provision and cli add live inside their CLI commands and exit the process themselves.

Changes

src/tools is the tools layer, with one entry (src/tools/index.ts): TOOL_REGISTRY, getTool, ToolId, and the console runners addMCPServerToClientsStep, removeMCPServerFromClientsStep, runDoctorReport, runProvision, runCliAdd and listSkills. Each runner resolves an exit code.
addMCPServerToClientsStep and removeMCPServerFromClientsStep move from src/shared/mcp-clients/install.ts to src/tools/mcp/console.ts. install.ts keeps only the per-client operations every MCP surface shares, and logs to the debug file instead of the UI.
Tool ids keep the program ids they had, so analytics program_id tags keep their values.
Each tool's TUI entry in src/tui/tools/<id> (doctor, mcp, slack) holds its flow and screens as a TuiTool, which getTuiTool finds and runTuiTool runs.
The CLI commands call the runners and exit through exitWith. wizard skill list is now the skill command's reserved list value.
The MCP prompt copy moves from JSON to a TS data module, mcp-role-prompts.copy.ts, with the same text less the Slack card, which moved in refactor: move every file into its layer #1375.
Focus on each runner's exit codes against the command it replaces. No program imports @tools.

Test plan

This tree does not compile alone. On the top of the stack (docs: bring docs, readmes and skills in line with the layers #1392): pnpm typecheck 0 errors, pnpm lint 0 errors, 236 test files (3,515 tests) pass, pnpm build ok.

Created with PostHog Desktop

github-actions · 2026-09-29T09:28:27Z

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

/wizard-ci all

Test all apps in a directory:

/wizard-ci ai-observability
/wizard-ci basic-integration
/wizard-ci feature-flags
/wizard-ci mcp-analytics
/wizard-ci replay-vision
/wizard-ci revenue
/wizard-ci self-driving
/wizard-ci warehouse
/wizard-ci warehouse-seeded

Test an individual app:

/wizard-ci ai-observability/anthropic
/wizard-ci ai-observability/google-adk
/wizard-ci ai-observability/groq

Show more apps

/wizard-ci ai-observability/manual-capture
/wizard-ci ai-observability/openai
/wizard-ci ai-observability/openai-agents
/wizard-ci ai-observability/opentelemetry
/wizard-ci ai-observability/vercel-ai
/wizard-ci basic-integration/android
/wizard-ci basic-integration/angular
/wizard-ci basic-integration/astro
/wizard-ci basic-integration/django
/wizard-ci basic-integration/fastapi
/wizard-ci basic-integration/flask
/wizard-ci basic-integration/flutter
/wizard-ci basic-integration/javascript-node
/wizard-ci basic-integration/javascript-web
/wizard-ci basic-integration/laravel
/wizard-ci basic-integration/next-js
/wizard-ci basic-integration/nuxt
/wizard-ci basic-integration/python
/wizard-ci basic-integration/rails
/wizard-ci basic-integration/react-native
/wizard-ci basic-integration/react-router
/wizard-ci basic-integration/sveltekit
/wizard-ci basic-integration/swift
/wizard-ci basic-integration/tanstack-router
/wizard-ci basic-integration/tanstack-start
/wizard-ci basic-integration/vue
/wizard-ci feature-flags/django
/wizard-ci feature-flags/next-js
/wizard-ci mcp-analytics/custom-dispatcher
/wizard-ci mcp-analytics/typescript-sdk
/wizard-ci replay-vision/javascript-node
/wizard-ci replay-vision/next-js
/wizard-ci replay-vision/react-vite
/wizard-ci revenue/stripe
/wizard-ci self-driving/astro
/wizard-ci self-driving/fastapi
/wizard-ci self-driving/nuxt
/wizard-ci self-driving/react-router
/wizard-ci self-driving/sveltekit
/wizard-ci warehouse/monorepo-env
/wizard-ci warehouse/multi-source-next
/wizard-ci warehouse/stripe-node
/wizard-ci warehouse/zero-source
/wizard-ci warehouse-seeded/next-stripe
/wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

/wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

johncwaters

Note

Automated review. Not written by a human.

johncwaters · 2026-10-02T13:27:57Z

+import { skillProgramOptions } from './skill-program-options';
+import type { Command } from './command';
+
+export const doctorCommand: Command = {


Note
Automated review. Not written by a human.

This command isn't wired in yet. main.ts:40 and src/cli/__tests__/wizard.test.ts:5 still import doctorCommand from ./src/tools/doctor/report, which no longer exports it. src/programs/posthog-doctor/index.ts:3 still imports POSTHOG_DOCTOR_PROGRAM, which this PR renames to POSTHOG_DOCTOR_FLOW. src/tools/cli-steering/__tests__/cli-add.test.ts still mocks @ui against the old runCliAdd(argv) signature. I assume a later PR in the stack fixes these. Can you confirm which one?

Handled later in the stack: the CLI moves to the new exports in #1385, src/programs/posthog-doctor is deleted in #1386, and the tests follow in #1390.

johncwaters · 2026-10-02T13:27:57Z

  },
  handler: (argv) => {
+    // `list` is the positional's one reserved value, not a skill id.
+    if (readSkillName(argv) === 'list') return exitWith(listSkills);


Note
Automated review. Not written by a human.

nit: skill list used to finish by letting the event loop drain. Through exitWith it now calls process.exit(0) right after the process.stdout.write calls. On macOS, writes to a piped stdout are async, so a long listing piped to a slow reader could get cut off. Setting process.exitCode for this path, instead of exiting, would avoid that.

johncwaters · 2026-10-02T13:27:57Z

  ProjectDataProfile,
 } from './mcp-project-profile';
-import copyData from './mcp-role-prompts.copy.json';
+import copyData from './mcp-role-prompts.copy';


Note
Automated review. Not written by a human.

nit: mcp-role-prompts.copy.json is still next to the new .copy.ts, and nothing imports it now. Its data matches the TS module exactly, minus the Slack card, so anyone who edits the JSON will see no effect. Delete it here?

Handled later in the stack: mcp-role-prompts.copy.json is deleted in #1386, leaving only the .copy.ts module.

johncwaters · 2026-10-02T13:27:57Z

+ * Any failure counts, not just a total wipeout: the step installs to every
+ * detected client, so one succeeding would otherwise mask the rest.
+ */
+export async function addMCPServerToClientsStep(


Note
Automated review. Not written by a human.

nit: the description asks reviewers to focus on each runner's exit codes, but no test pins them. Nothing covers addMCPServerToClientsStep (1 on any failure or no clients), removeMCPServerFromClientsStep, runDoctorReport, runProvision or listSkills. They now take an injected log and resolve a number, so a small table test per runner is cheap and would stop the codes drifting later in the stack.

Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589

edwinyjlim

Review of #1384 at 81ab2073. Items outside this PR's diff lines:

[P3] A tools-layer test still drives the CLI command
src/tools/cli-steering/__tests__/cli-add.test.ts:54

It imports cliAddCommand from @cli/commands/cli/add. The PR's own rule is that tools never import the CLI, and #1391's references make this TS6307. It compiles here only because the stack lands together.

Suggested fix: If reviewed in isolation, call runCliAdd directly with a sink stub.

Drafted with Claude Code from a review of the whole Release C stack, re-verified at the stack tip 2e31e438. Line numbers are at this PR's head.

edwinyjlim · 2026-10-05T09:15:30Z

+      );
+      return;
+    }
+    withSignals(async (signal) => {


[P2] wizard doctor in a non-TTY loses its error code and message

Same root cause as exitWith in #1383. doctor is the one tool command with no catch around runTuiTool, so a non-TTY run gets a bare PHW_INTERNAL_UNHANDLED line. slack.ts:36-48 and mcp/tutorial.ts:36-48 handle the same rejection; mcp/add.ts:70-73 falls back to the console runner.

Suggested fix: Prefer fixing exitWith. If it stays as is, give doctor.ts the slack.ts shape: catch, log the error, emit CliInteractiveRequired, return 1.

gewenyu99 · 2026-10-05T09:48:03Z

Folded into #1378 so the phase builds green PR by PR. This PR keeps its review; the code lands through #1378.

This was referenced Sep 29, 2026

refactor: run a program on the caller's session store #1379

Closed

refactor: route the cli, e2e harness and examples through public entries #1385

Closed

gewenyu99 added this pull request to stack #1394 September 29, 2026 09:41

gewenyu99 force-pushed the posthog/functional-r9-11-split-the-tools-out-of-programs branch from f8c053f to f0d8c36 Compare September 30, 2026 08:06

gewenyu99 force-pushed the posthog/functional-r9-10-add-the-tui-and-headless-hosts-and-let-o branch from abb8407 to 699ef88 Compare September 30, 2026 08:06

gewenyu99 removed this pull request from stack #1394 September 30, 2026 08:07

gewenyu99 added this pull request to stack #1400 September 30, 2026 08:07

gewenyu99 force-pushed the posthog/functional-r9-11-split-the-tools-out-of-programs branch from f0d8c36 to c03101f Compare September 30, 2026 08:44

gewenyu99 force-pushed the posthog/functional-r9-10-add-the-tui-and-headless-hosts-and-let-o branch from 699ef88 to 7e6425d Compare September 30, 2026 08:44

gewenyu99 marked this pull request as ready for review September 30, 2026 09:17

gewenyu99 requested a review from a team as a code owner September 30, 2026 09:17

gewenyu99 force-pushed the posthog/functional-r9-10-add-the-tui-and-headless-hosts-and-let-o branch from 7e6425d to a5ec738 Compare September 30, 2026 14:11

gewenyu99 force-pushed the posthog/functional-r9-11-split-the-tools-out-of-programs branch from c03101f to 24a0712 Compare September 30, 2026 14:11

This was referenced Sep 30, 2026

refactor: add the session store, run contract and run helpers #1376

Merged

refactor: move the agent, programs, tui, hosts, tools and cli onto the session store #1378

Merged

gewenyu99 force-pushed the posthog/functional-r9-10-add-the-tui-and-headless-hosts-and-let-o branch from a5ec738 to 4f429c7 Compare October 1, 2026 06:43

gewenyu99 force-pushed the posthog/functional-r9-11-split-the-tools-out-of-programs branch from 24a0712 to 95d85e8 Compare October 1, 2026 06:43

gewenyu99 removed this pull request from stack #1400 October 1, 2026 06:47

gewenyu99 added this pull request to stack #1403 October 1, 2026 06:47

johncwaters approved these changes Oct 2, 2026

View reviewed changes

gewenyu99 force-pushed the posthog/functional-r9-10-add-the-tui-and-headless-hosts-and-let-o branch from 4f429c7 to 6086517 Compare October 2, 2026 16:47

gewenyu99 force-pushed the posthog/functional-r9-11-split-the-tools-out-of-programs branch from 95d85e8 to 4be5b3f Compare October 2, 2026 16:47

gewenyu99 force-pushed the posthog/functional-r9-10-add-the-tui-and-headless-hosts-and-let-o branch from 6086517 to 3fe2559 Compare October 2, 2026 20:50

gewenyu99 force-pushed the posthog/functional-r9-11-split-the-tools-out-of-programs branch from 4be5b3f to 1b7941f Compare October 2, 2026 20:50

refactor: split the tools out of programs

81ab207

Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589

gewenyu99 force-pushed the posthog/functional-r9-11-split-the-tools-out-of-programs branch from 1b7941f to 81ab207 Compare October 3, 2026 20:40

gewenyu99 force-pushed the posthog/functional-r9-10-add-the-tui-and-headless-hosts-and-let-o branch from 3fe2559 to 216a0dd Compare October 3, 2026 20:40

edwinyjlim reviewed Oct 5, 2026

View reviewed changes

edwinyjlim approved these changes Oct 5, 2026

View reviewed changes

gewenyu99 removed this pull request from stack #1403 October 5, 2026 09:47

gewenyu99 closed this Oct 5, 2026

Navigation Menu

Repository navigation

refactor: split the tools out of programs - #1384

refactor: split the tools out of programs#1384
gewenyu99 wants to merge 1 commit into
posthog/functional-r9-10-add-the-tui-and-headless-hosts-and-let-ofrom
posthog/functional-r9-11-split-the-tools-out-of-programs

gewenyu99 commented Sep 29, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 29, 2026

Uh oh!

johncwaters left a comment

Uh oh!

johncwaters Oct 2, 2026

Uh oh!

gewenyu99 Oct 2, 2026

Uh oh!

johncwaters Oct 2, 2026

Uh oh!

johncwaters Oct 2, 2026

Uh oh!

gewenyu99 Oct 2, 2026

Uh oh!

johncwaters Oct 2, 2026

Uh oh!

edwinyjlim left a comment

Uh oh!

edwinyjlim Oct 5, 2026

Uh oh!

gewenyu99 commented Oct 5, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

gewenyu99 commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

Test plan

Uh oh!

github-actions Bot commented Sep 29, 2026

🧙 Wizard CI

Uh oh!

johncwaters left a comment

Choose a reason for hiding this comment

Uh oh!

johncwaters Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

gewenyu99 Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

johncwaters Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

johncwaters Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

gewenyu99 Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

johncwaters Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

edwinyjlim left a comment

Choose a reason for hiding this comment

Uh oh!

edwinyjlim Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

gewenyu99 commented Oct 5, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gewenyu99 commented Sep 29, 2026 •

edited

Loading