feat(menu): describe skills with title, description and tags by edwinyjlim · Pull Request #447 · PostHog/context-mill

edwinyjlim · 2026-10-29T00:21:59Z

Summary

Every skill entry and bundle variant in the skill menu carries a title, a description and tags, so the wizard can pick skills by tag. A bundle's own entry carries tags only. Wizards already in the field ignore the new fields and get the same files as before. The context/agents/ README points to the wizard's own copy of the prompts.

Stack

The five PRs, in merge order:

PR	What it does	How it fits in
Menu metadata · #447 (this PR)	Adds a title, description and tags to every skill-menu entry.	Ships in a context-mill release. PR 1 reads the tags, and resolves the same without them.
PR 1 · Skill resolver · PostHog/wizard#1551	Adds the `wizard-agent-dag` flag and the shared skill resolver the orchestrator uses when it is on.	Base of the wizard stack, on `main`. Every later PR hangs its new path off this flag.
PR 2 · Project traits · PostHog/wizard#1552	Adds the project traits service and hands the orchestrator the traits skills match on.	On PR 1. Supplies the resolver's match keys. Nothing reads a trait beyond the framework yet.
PR 3 · Queue from frontmatter · PostHog/wizard#1553	Queues a flow that has no planner prompt from the graph its prompts' frontmatter declares.	On PR 2. No flow lacks a planner until PR 4 bundles one.
PR 4 · Prompts in the wizard · PostHog/wizard#1554	Bundles each orchestrator flow's prompts in its program. With the flag on, runs read them.	Top of the stack, on PR 3. posthog-integration's folder has no planner prompt, so PR 3's queue runs it.

The four wizard PRs ship in one release with wizard-agent-dag off. This PR needs none of them, and none of them needs this one released first.

Menu metadata

flowchart LR
  sources["Skill sources and config"] --> build["Build"]
  build --> menu["skill-menu.json<br/>entries and bundle variants<br/>with title, description, tags"]
  build -.-> warn["Warning for each skill<br/>that has no tags"]
  menu --> wizard["New wizard<br/>picks skills by tag"]
  menu --> older["Wizards in the field<br/>ignore the new fields"]

An entry's tags are its skill's own tags plus:

Tag	Added when
`framework:<framework>`	the skill serves a framework
its group, such as `integration`	the skill belongs to a group
`default`	the skill is its group's default variant

Two entries from this build:

Entry	Title	Tags
`integration-nextjs-app-router`	Next.js App Router	`nextjs` `react` `ssr` `app-router` `javascript` `javascript_web` `javascript_node` `framework:nextjs` `integration` `default`
`integration-django`	Django	`django` `python` `framework:django` `integration`

Build checks

Check	Fails when
`npm run check:bundle-diff -- <baseline-dist> [dist]`	two builds differ in file contents beyond `title`, `description` and `tags`, or in which files are executable
Old-wizard compatibility test	the wizard's skill resolution (`expandBundleEntry`, `resolveSkillVariantId`, the spellbook's `selectSkills`) picks differently from the snapshot of `main`'s build (49 prompts, 1,145 variant resolutions, 386 spellbook selections), or a flow stops publishing exactly one planner prompt

The build also warns, without failing, about skills that have no tags.

Agent prompts

The wizard bundles its own copy of the agent prompts at src/programs/<id>/agent/. The copy in context/agents/ is published for wizard versions that fetch their prompts, and its README asks for a prompt change in both places.

Test plan

npm test: 30 files, 329 tests pass. The compat snapshot regenerated from main's code at 1.64.0 is byte-identical to the committed one
check:bundle-diff between clean builds of main at 1.64.0 and this branch, from one docs cache: no differences beyond title, description and tags across 2,976 files, so every SKILL.md, reference and agent prompt is byte-identical
After release: a current npm wizard smoke run against the new latest

🤖 Generated with Claude Code

…agent prompts Every skill-menu entry and bundle variant carries `title`, `description` and `tags`. The tags are the skill's own tags plus `framework:<framework>` for the framework it serves, its group, and `default` when set. The wizard selects skills by these tags; old wizards ignore the new fields. Agent prompts in `context/agents/` are frozen: the wizard bundles its own copy, and this one keeps publishing for wizard versions already in the field. `scripts/check-agents-frozen.sh` fails a pull request that edits it. `check:bundle-diff` compares two builds and fails on any difference beyond the menu metadata. The old-wizard compatibility test replays the wizard's skill resolution and checks that agent assets, skill variants and spellbook selections stay the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-10-29T00:22:10Z

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

/wizard-ci all

Test all apps in a directory:

/wizard-ci ai-observability
/wizard-ci basic-integration
/wizard-ci feature-flags
/wizard-ci mcp-analytics
/wizard-ci replay-vision
/wizard-ci revenue
/wizard-ci self-driving
/wizard-ci warehouse
/wizard-ci warehouse-seeded

Test an individual app:

/wizard-ci ai-observability/anthropic
/wizard-ci ai-observability/google-adk
/wizard-ci ai-observability/groq

Show more apps

/wizard-ci ai-observability/manual-capture
/wizard-ci ai-observability/openai
/wizard-ci ai-observability/openai-agents
/wizard-ci ai-observability/opentelemetry
/wizard-ci ai-observability/vercel-ai
/wizard-ci basic-integration/android
/wizard-ci basic-integration/angular
/wizard-ci basic-integration/astro
/wizard-ci basic-integration/django
/wizard-ci basic-integration/fastapi
/wizard-ci basic-integration/flask
/wizard-ci basic-integration/flutter
/wizard-ci basic-integration/javascript-node
/wizard-ci basic-integration/javascript-web
/wizard-ci basic-integration/laravel
/wizard-ci basic-integration/next-js
/wizard-ci basic-integration/nuxt
/wizard-ci basic-integration/python
/wizard-ci basic-integration/rails
/wizard-ci basic-integration/react-native
/wizard-ci basic-integration/react-router
/wizard-ci basic-integration/sveltekit
/wizard-ci basic-integration/swift
/wizard-ci basic-integration/tanstack-router
/wizard-ci basic-integration/tanstack-start
/wizard-ci basic-integration/vue
/wizard-ci feature-flags/django
/wizard-ci feature-flags/next-js
/wizard-ci mcp-analytics/custom-dispatcher
/wizard-ci mcp-analytics/typescript-sdk
/wizard-ci replay-vision/javascript-node
/wizard-ci replay-vision/next-js
/wizard-ci replay-vision/react-vite
/wizard-ci revenue/stripe
/wizard-ci self-driving/astro
/wizard-ci self-driving/fastapi
/wizard-ci self-driving/nuxt
/wizard-ci self-driving/react-router
/wizard-ci self-driving/sveltekit
/wizard-ci warehouse/monorepo-env
/wizard-ci warehouse/multi-source-next
/wizard-ci warehouse/stripe-node
/wizard-ci warehouse/zero-source
/wizard-ci warehouse-seeded/next-stripe
/wizard-ci warehouse-seeded/next-stripe-declined

Test against a wizard branch:

/wizard-ci all wizard:my-branch

Add wizard:<branch> to any command above to pin the wizard branch. It defaults to main.

Results will be posted here when complete.

…verride label The agents-frozen check runs in its own workflow, so adding or removing the label re-runs only that check. With the label it passes with a warning to make the same fix in the wizard's copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The context/agents README says the wizard bundles its own copy of the prompts and that a prompt change goes in both places; review keeps the two in step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude <noreply@anthropic.com>

johncwaters

Note

Automated review. Not written by a human.

Verdict: APPROVE WITH NITS

See inline comments.

sarahxsanders

fields themselves are straightforward

same concern here about prompts. bundled prompts in an older wizard will still depend on names in the latest menu

how should we preserve names across future releases?

The compatibility test fails when a flow stops publishing exactly one planner prompt, which a wizard that fetches its prompts needs. The menu writer test runs the real config, so a bundled group's own tags reach its menu entry. The bundle diff's instructions build both sides from the same docs cache and from checkouts with no generated example files. Co-Authored-By: Claude <noreply@anthropic.com>

gewenyu99

Initial comments

A bundle's menu entry took its title and description from the first variant the build saw, so one framework's copy named a group serving forty. The entry now carries tags only, and each variant keeps its own. The bundle diff also compares executable bits and names a missing build folder. The compatibility snapshot's header says entries are only added. Co-Authored-By: Claude <noreply@anthropic.com>

This was referenced Oct 29, 2026

feat(orchestrator): pick task skills by project traits, in shadow PostHog/wizard#1497

Closed

feat(programs): programs own their orchestrator flows and prompts PostHog/wizard#1498

Closed

edwinyjlim and others added 3 commits October 29, 2026 12:49

ci(agents): pass the base ref to the freeze check through env

65d57c1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ci(agents): drop the agent prompt freeze check

619cf5f

The context/agents README says the wizard bundles its own copy of the prompts and that a prompt change goes in both places; review keeps the two in step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

edwinyjlim changed the title ~~feat(menu): describe skills with title, description and tags; freeze agent prompts~~ feat(menu): describe skills with title, description and tags Oct 29, 2026

edwinyjlim mentioned this pull request Oct 29, 2026

feat(traits): write the run's framework through its project trait PostHog/wizard#1499

Closed

4 of 6 tasks

edwinyjlim and others added 2 commits November 11, 2026 15:17

Merge remote-tracking branch 'origin/main' into dag/c1-menu-metadata

9e19fb1

test(menu): regenerate the old-wizard snapshot on current main

689f06d

Co-Authored-By: Claude <noreply@anthropic.com>

edwinyjlim marked this pull request as ready for review November 12, 2026 17:54

edwinyjlim requested a review from a team as a code owner November 12, 2026 17:54

johncwaters reviewed Nov 12, 2026

View reviewed changes

Comment thread scripts/lib/tests/old-wizard-compat.test.js Resolved

Comment thread scripts/check-bundle-diff.js Resolved

sarahxsanders approved these changes Nov 12, 2026

View reviewed changes

gewenyu99 reviewed Nov 13, 2026

View reviewed changes

Comment thread scripts/lib/build-phases.js Resolved

Comment thread scripts/build.js Resolved

Comment thread scripts/check-bundle-diff.js Resolved

Comment thread scripts/check-bundle-diff.js Resolved

Comment thread scripts/lib/tests/old-wizard-compat.test.js Resolved

gewenyu99 approved these changes Nov 13, 2026

View reviewed changes

edwinyjlim merged commit 8a38619 into main Nov 13, 2026
17 checks passed

releaser-wizard Bot mentioned this pull request Nov 13, 2026

chore(main): release 1.65.0 #513

Merged

Navigation Menu

Repository navigation

feat(menu): describe skills with title, description and tags - #447

feat(menu): describe skills with title, description and tags#447
edwinyjlim merged 8 commits into
mainfrom
dag/c1-menu-metadata

edwinyjlim commented Oct 29, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 29, 2026

Uh oh!

johncwaters left a comment

Uh oh!

Uh oh!

Uh oh!

sarahxsanders left a comment

Uh oh!

gewenyu99 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

edwinyjlim commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Stack

Menu metadata

Build checks

Agent prompts

Test plan

Uh oh!

github-actions Bot commented Oct 29, 2026

🧙 Wizard CI

Uh oh!

johncwaters left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sarahxsanders left a comment

Choose a reason for hiding this comment

Uh oh!

gewenyu99 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

edwinyjlim commented Oct 29, 2026 •

edited

Loading