Skip to content

refactor(skills): stop reading bundle entries from the skill menu - #1568

Merged
edwinyjlim merged 6 commits into
dag/8-traits-and-tags-onlyfrom
dag/9-drop-bundles
Nov 17, 2026
Merged

edwinyjlim merged 6 commits into
dag/8-traits-and-tags-onlyfrom
dag/9-drop-bundles

Conversation

@edwinyjlim

@edwinyjlim edwinyjlim commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Behaviour

The wizard installs a skill from its own zip, and from nothing else.

  • A menu entry that is a bundle of variants is left out when the menu is fetched. It is never listed by load_skill_menu, resolved, or installed.
  • A category that holds only bundles is left out with them, so load_skill_menu lists no empty category.
  • A menu that still lists a bundle entry does not break the fetch. Every other entry is read as before.

The bundle installer, the bundle and variants fields on a menu entry, and the expansion of a bundle's variants are gone.

Verified

  • pnpm typecheck and pnpm lint (0 errors).
  • pnpm test: 292 files, 4,474 tests pass, 1 skipped.
  • skill-menu.test.ts covers a menu with a bundle entry: the entry is left out, its neighbour is kept, and a category of bundles alone is gone.
  • flow-graphs.test.ts still finds every skill the bundled prompts name on the recorded context-mill 1.66.0 menu, with its five bundle entries left out.
  • Two local runs of this branch against the published context-mill 1.66.0 release, whose menu lists five bundle entries: the posthog-integration flow and the error-tracking flow, each on a Next.js app. Every task finished or was correctly skipped, and no task failed. The error-tracking tasks listed the error-tracking menu category and installed the skill they chose from it.

Rollout

This is the last of four stacked PRs that remove the old code paths. They merge together and ship in one release.

  • They wait until wizard-agent-dag and wizard-agent-dag-skills have both held at 100% with no stop rule tripped, no run that fetched its prompts, and no run that fell back to an old skill.
  • Both flags stay on in PostHog after the release. Released versions still read them, and a deleted flag reads as off.
  • Cloud moves to the release afterwards, with its context-mill pin at 1.66.0 or later.

context-mill publishes bundle entries until it releases without the old step skills. Until then, a wizard with this change does not do four things:

  • install a skill named by its old family, such as integration-v2-install, through install_skill. On main that name is already not found
  • install a variant id from the integration-v2/* and replay-vision-enable-replay menu categories. No bundled prompt names one
  • install replay-vision-enable-replay, which the published replay-vision-setup skill names for its session replay step. That skill is read by wizard skill replay-vision-setup and by a replay-vision run on the linear sequence. wizard replay-vision on the orchestrator names integration-session-replay and is not affected
  • hand over the old step skills in the spell book of a posthog-integration run with no detected framework

No orchestrator flow reaches a bundle: every skill its prompts name is published as its own zip.

All four end with that context-mill release, in which replay-vision-setup names integration-session-replay. The oldest context-mill release the wizard accepts moves up to it afterwards.

🤖 Generated with Claude Code

A menu entry that is a bundle of variants is left out when the menu is
fetched, so it is never offered or installed. The bundle installer, the
bundle fields on a menu entry and the expansion of a bundle's variants
are removed.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@edwinyjlim
edwinyjlim added this pull request to stack #1569 November 17, 2026 01:22
@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci feature-flags
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci feature-flags/django
  • /wizard-ci feature-flags/next-js
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/go-sdk
  • /wizard-ci mcp-analytics/python-sdk
  • /wizard-ci mcp-analytics/ruby-sdk
  • /wizard-ci mcp-analytics/rust-sdk
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

@edwinyjlim
edwinyjlim marked this pull request as ready for review November 17, 2026 17:09
@edwinyjlim
edwinyjlim requested a review from a team as a code owner November 17, 2026 17:09
@veria-ai

veria-ai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

PR overview

No security issues were found in this pull request.

Reviews

@posthog-security-review-bot posthog-security-review-bot Bot added security-review Request a security review and removed security-review Request a security review labels Nov 17, 2026
@posthog-security-review-bot

posthog-security-review-bot Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

✅ Security review complete — 9da9a9533589

No findings above the confidence threshold, in 5 min.

Add the security-review label to run again on the latest commit.

@posthog-security-review-bot posthog-security-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent-driven security review - no findings.

@posthog-security-review-bot posthog-security-review-bot Bot removed the security-review Request a security review label Nov 17, 2026

@gewenyu99 gewenyu99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥸 Reviewed by NotVincent, totally not Vincent. The real Vincent will review it separately. Probably slop, please disregard.

Comment thread src/shared/skill-menu.ts Outdated
export function prepareSkillMenu(menu: SkillMenu): SkillMenu {
for (const [category, entries] of Object.entries(menu.categories)) {
const family = skillFamily(category);
menu.categories[category] = entries

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this breaks replay-vision on v1.66.0, the current context-mill floor, because:

  • on v1.66.0, replay-vision-enable-replay only ships as a bundle entry
  • this filter drops bundle entries, so wizard skill replay-vision-setup and linear replay-vision can't find it
  • the categories whose only entries are bundles stay as empty arrays and still show up in load_skill_menu

Bundle entries in the live v1.66.0 skill-menu.json:

integration-v2/capture              integration-v2-capture
integration-v2/error-tracking-step  integration-v2-error-tracking-step
integration-v2/init                 integration-v2-init
integration-v2/install              integration-v2-install
replay-vision-enable-replay         replay-vision-enable-replay

Maybe delete a category key once its list is empty, and either bump CONTEXT_MILL_FLOOR once a context-mill release without bundles ships, or keep reading bundles until then.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right on all three points, and the gap is temporary. On 1.66.0 the published replay-vision-setup skill names replay-vision-enable-replay, which ships only as a bundle, so wizard skill replay-vision-setup and a linear replay-vision run can't install that step. wizard replay-vision on the orchestrator names integration-session-replay and is not affected.

It closes with the next context-mill release (PostHog/context-mill#537), where replay-vision-setup names integration-session-replay and the bundle is gone. That release follows this one, so we're accepting the window and not keeping the bundle reader for it. The description now lists this case.

  • Empty categories: fixed in f69248f. A category that holds only bundles is left out of the menu.
  • CONTEXT_MILL_FLOOR: it moves up once that context-mill release is out.

edwinyjlim and others added 5 commits November 17, 2026 09:43
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A category whose entries are all bundles of variants was left as an empty
list, so load_skill_menu listed a category with nothing in it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@edwinyjlim
edwinyjlim merged commit 5b30ce2 into main Nov 17, 2026
19 checks passed
@edwinyjlim
edwinyjlim deleted the dag/9-drop-bundles branch November 17, 2026 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants