Repository navigation
feat(programs): bundle each orchestrator flow's agent prompts - #1554
Merged
edwinyjlim merged 16 commits intoNov 13, 2026
Merged
Conversation
A .md import is the file's text. tsdown inlines it in the build, a Vitest plugin loads it in tests, and a Node hook loads it in a run from source: pnpm try and the TUI host the snapshot and wizard-ci scripts spawn. Co-Authored-By: Claude <noreply@anthropic.com>
Each orchestrator program keeps its task prompts in agent/<type>.md, and its agent-prompts.ts imports them as text. With wizard-agent-dag on, the orchestrator reads the bundled prompts; off, it fetches context-mill's copy of the flow. The files match context-mill 1.64.0 byte for byte, with two exceptions in posthog-integration: its folder has no planner prompt, so a run with the flag on is queued from the frontmatter, and capture waits for mcp-analytics, an edge the planner decides case by case. The scanner smoke test, which every build runs, scans each agent/ folder. pnpm visual-dags draws every program's graph from its frontmatter. Co-Authored-By: Claude <noreply@anthropic.com>
This was referenced Nov 12, 2026
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Test against a Context Mill branch:
Add Results will be posted here when complete. |
2 of 3 tasks
edwinyjlim
marked this pull request as ready for review
November 12, 2026 17:54
edwinyjlim
requested review from
JamesPatrickGill,
TueHaulund,
ablaszkiewicz,
fasyy612,
hpouillot and
ksvat
and removed request for
a team
November 12, 2026 17:54
posthog-security-review-bot
Bot
added
the
security-review
|
✅ Security review complete — 1 finding posted as a review, in 7 min. Add the |
johncwaters
reviewed
Nov 12, 2026
johncwaters
left a comment
johncwaters
left a comment
Contributor
There was a problem hiding this comment.
Note
Automated review. Not written by a human.
Verdict: APPROVE
sarahxsanders
approved these changes
Nov 12, 2026
sarahxsanders
left a comment
sarahxsanders
left a comment
Collaborator
There was a problem hiding this comment.
a littttle bit worried about creating two copies of the prompts/relying on both being updated here. how can we enforce that they are in sync?
is this temporary/should we discuss if we've outgrown the need for separate context + wizard repos?
![posthog-security-review-bot[bot]](https://avatars.githubusercontent.com/in/4551195?s=60&v=4){kind=small}
posthog-security-review-bot
Bot
removed
the
security-review
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
|
✅ Security review complete — No findings above the confidence threshold, in 6 min. Add the |
posthog-security-review-bot
Bot
removed
the
security-review
gewenyu99
reviewed
Nov 13, 2026
# Conflicts: # scripts/wizard-ci-mcp.no-jest.ts
…kill A bundled prompt can name a skill the menu no longer publishes. The run still fails before any task starts, and it now says to run the latest wizard, since trying again cannot help. Fetched prompts, a menu that did not load and a run with no framework keep the message that asks to try again. Co-Authored-By: Claude <noreply@anthropic.com>
# Conflicts: # src/agent/runner/sequence/README.md # src/agent/runner/sequence/orchestrator/__tests__/frontmatter-queue.test.ts
…ssing planner its own failure A prompt bundle that fails to load is reported as an exception with the step `orchestrator_bundled_prompts`, and the run fetches its flow from context-mill. A fetched flow with no planner prompt ends the run with PHW_AGENT_ORCHESTRATOR_NO_PLANNER and a message for the user. The README says the `orchestrator_prompts` tag is on every event sent after the prompts load. Co-Authored-By: Claude <noreply@anthropic.com>
… spells its flags The parser reads a prompt with no frontmatter as a task with no dependencies, and any flag value but `true` as false. The check covers every program that bundles prompts, recorded or not. Co-Authored-By: Claude <noreply@anthropic.com>
…rator runs them `pnpm visual-dags` draws the sink in the last wave, and a task the wizard queues itself that names no dependency in the wave before it. A missing page opener no longer fails the script after the page is written. Co-Authored-By: Claude <noreply@anthropic.com>
gewenyu99
approved these changes
Nov 13, 2026
This was referenced Nov 13, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Each orchestrator program keeps its task prompts in
src/programs/<id>/agent/<type>.md. Withwizard-agent-dagon, the orchestrator reads these bundled prompts. With the flag off or unset, it fetches context-mill's copy of the flow, as today.posthog-integration's folder has no planner prompt. With the flag on, its queue comes from the graph its frontmatter declares, and no planner agent runs. The other five programs keep their planners. A fetched flow with no planner prompt fails the run with
PHW_AGENT_ORCHESTRATOR_NO_PLANNER, flag on or off.Stack
The five PRs, in merge order:
wizard-agent-dagflag and the shared skill resolver the orchestrator uses when it is on.main. Every later PR hangs its new path off this flag.The four wizard PRs merge in that order and ship in one release with
wizard-agent-dagoff. A run with the flag off or unset behaves as it does onmain.Where a run's prompts come from
flowchart LR subgraph wizard["Wizard release"] md["src/programs/<id>/agent/*.md"] --> mod["agent-prompts.ts<br/>imports each file as text"] mod --> cfg["ProgramConfig.agentPrompts"] end cm["context-mill release<br/>agent menu and prompts"] cfg -- "wizard-agent-dag on" --> registry["Orchestrator's prompt registry"] cm -- "wizard-agent-dag off" --> registryA bundle that fails to load is reported as an exception with the step
orchestrator_bundled_prompts, and the run fetches instead.Every event sent after the prompts load carries which one the run used as the
orchestrator_promptstag:bundledorcontext-mill. That tag says which path a run took. The flag's value on an event does not: a pinned run reports the flag as on and still fetches.integration-v2error-trackingreplay-visionfeature-flagsmetricssupportWhat differs from context-mill 1.65.0
The bundled files match the recorded release byte for byte, with two exceptions, both in posthog-integration:
integrate-posthog.md, the planner promptcaptureaddsmcp-analyticsto itsdependsOnmcp-analytics.posthog-integration's graph, flag on
Edges a longer path already implies are left out.
pnpm visual-dagsdraws every program's graph from its frontmatter, in the waves the orchestrator runs: the sink last, and a task the wizard queues itself just before it.What a flag-on posthog-integration run does differently
excludedTaskTypesandseedTaskslabelcaptureandmcp-analyticscapturewaitsPrompts and skills
With the flag on, a run's prompts come from its wizard release, and its skills still install from context-mill. A bundled prompt names its skills by menu id or by family, so every name must stay on the menu.
flow-graphs.test.tschecks the names against the recorded menu.A local context-mill (
--local-context-mill) serves the skills either way, and the prompts only with the flag off.When a bundled prompt names a skill the loaded menu has under no id and no family, the wizard is older than the skills. The run fails before any task starts, as it does for any missing skill, and the message says to run the latest wizard. Fetched prompts, a menu that did not load and a run with no framework keep the message that asks to try again, so a flag-off run reads as before.
A run pinned to a context-mill release with
--context-mill-versionfetches its prompts and its skills from that release, flag on or off.Loading Markdown
A
.mdimport is the file's text, declared intypes/markdown.d.ts.loader: { '.md': 'text' }. Each program's prompts land in their own chunk, loaded on first use.markdownAsText.scripts/markdown-loader.no-jest.mjs, passed totsxbypnpm try, the snapshot script and the e2e harness's host launcher.Checks
src/programs/__tests__/agent-prompts.test.tsagent/folder doesn't load it, a file in the folder isn't imported, a prompt has no frontmatter or a flag that is nottrueorfalse, or a prompt differs from the recorded release beyond the two differences abovesrc/agent/runner/sequence/orchestrator/__tests__/flow-graphs.test.tsdependsOnnames a task outside its folder, a folder has no single sink or cannot be ordered, askillsentry is missing from the recorded menu, or posthog-integration's graph queues a different task setpnpm test:warlock, run by every buildagent/folder as a fetched skill is scanned.pnpm test:smoke, run by every buildTest plan
pnpm typecheckpnpm lintpnpm build: the prompts are inlined in six lazy chunks, and the scanner smoke test passes on all six folderspnpm exec vitest run: 288 files passfrontmatter-queue.test.ts: a bundle that fails to load falls back to the fetch, and a fetched flow with no planner fails with its own code🤖 Generated with Claude Code