Skip to content

refactor(orchestrator): read agent prompts from the wizard only - #1565

Merged
edwinyjlim merged 2 commits into
mainfrom
dag/6-bundled-prompts-only
Nov 17, 2026
Merged

edwinyjlim merged 2 commits into
mainfrom
dag/6-bundled-prompts-only

Conversation

@edwinyjlim

@edwinyjlim edwinyjlim commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Behaviour

An orchestrator run reads its agent prompts from the wizard, and nowhere else.

  • The registry is built from the prompts the program bundles in agent/. A flow with a seed: true prompt keeps its planner. A flow without one is queued from the graph its frontmatter declares.
  • A program that bundles no prompts, or whose bundle fails to load, fails the run with PHW_AGENT_ORCHESTRATOR_NO_PLANNER. A failed load is also reported as an exception with the step orchestrator_bundled_prompts.
  • Skills are picked by resolveSkill with the project's match keys.
  • A run pinned with --context-mill-version reads the bundled prompts like any other run. The pin selects skills only.

The wizard-agent-dag flag, the fetch of agent-menu.json and agents-*.md, the runner's own menu lookup and the orchestrator_prompts tag are gone. The recorded copy of context-mill's prompts under fixtures/context-mill/ goes with them, and record:context-mill-fixtures records the skill menu only.

Verified

  • pnpm typecheck and pnpm lint (0 errors).
  • pnpm test: 291 files, 4,454 tests pass, 1 skipped.
  • frontmatter-queue.test.ts covers the two new failures: a bundle that fails to load, and a program that bundles none.
  • flow-graphs.test.ts and agent-prompts.test.ts still check every bundled flow: each dependsOn names a task in its folder, each folder has one sink and can be ordered, and each skills entry is on the recorded menu.

Rollout

This is the first of four stacked PRs that remove the old code paths. They merge together and ship in one release.

  • They wait until wizard-agent-dag and wizard-agent-dag-skills have both held at 100% with no stop rule tripped, no run that fetched its prompts, and no run that fell back to an old skill.
  • Both flags stay on in PostHog after the release. Released versions still read them, and a deleted flag reads as off.
  • Cloud moves to the release afterwards, with its context-mill pin at 1.66.0 or later.

🤖 Generated with Claude Code

The orchestrator builds its registry from the prompts a program bundles
and picks skills with the shared resolver and the project's match keys.
The wizard-agent-dag flag, the remote prompt fetch and the old menu
lookup are removed. A program that bundles no prompts, or whose bundle
fails to load, fails the run with PHW_AGENT_ORCHESTRATOR_NO_PLANNER.

A run pinned with --context-mill-version reads the bundled prompts too;
the pin selects skills only.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci feature-flags
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci feature-flags/django
  • /wizard-ci feature-flags/next-js
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/go-sdk
  • /wizard-ci mcp-analytics/python-sdk
  • /wizard-ci mcp-analytics/ruby-sdk
  • /wizard-ci mcp-analytics/rust-sdk
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

@edwinyjlim
edwinyjlim added this pull request to stack #1569 November 17, 2026 01:22

@johncwaters johncwaters left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES

See inline comments.

Automated review by Glimmervoid. Not written by a human

@gewenyu99 gewenyu99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥸 Reviewed by NotVincent, totally not Vincent. The real Vincent will review it separately. Probably slop, please disregard.

@edwinyjlim
edwinyjlim marked this pull request as ready for review November 17, 2026 17:08
@edwinyjlim
edwinyjlim requested a review from a team as a code owner November 17, 2026 17:08
@veria-ai

veria-ai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

PR overview

No security issues were found in this pull request.

Reviews

@posthog-security-review-bot posthog-security-review-bot Bot added the security-review Request a security review label Nov 17, 2026
@posthog-security-review-bot

posthog-security-review-bot Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

✅ Security review complete — 0085c79b0364

No findings above the confidence threshold, in 6 min.

Add the security-review label to run again on the latest commit.

@posthog-security-review-bot posthog-security-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent-driven security review - no findings.

@posthog-security-review-bot posthog-security-review-bot Bot removed the security-review Request a security review label Nov 17, 2026
…and every framework resolves its skills

A program that runs the orchestrator and bundles no prompts fails the suite,
since its runs would fail. On the recorded menu, every framework resolves a
framework pack and every skill a bundled flow names.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@edwinyjlim
edwinyjlim merged commit 3423b5e into main Nov 17, 2026
19 checks passed
@edwinyjlim
edwinyjlim deleted the dag/6-bundled-prompts-only branch November 17, 2026 20:43
edwinyjlim added a commit that referenced this pull request Nov 17, 2026
main holds #1565 and #1566 as squash commits, which this branch already
contains commit by commit. The merge takes main's one other change, #1562.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants