Skip to content

feat(orchestrator): read the step skills by name and complete them from the pack - #1566

Merged
edwinyjlim merged 11 commits into
dag/6-bundled-prompts-onlyfrom
dag/7-repackaged-skills-only
Nov 17, 2026
Merged

edwinyjlim merged 11 commits into
dag/6-bundled-prompts-onlyfrom
dag/7-repackaged-skills-only

Conversation

@edwinyjlim

@edwinyjlim edwinyjlim commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Behaviour

A task reads the step skills by their own names, and a step skill written once for every framework is always completed from the framework's pack.

  • The bundled prompts name the integration-* step skills, such as integration-install and integration-session-replay.
  • A skill whose menu entry has frameworkReferences is installed once a run into an emptied folder, completed from the framework's pack, and its rewritten SKILL.md is scanned.
  • When the skill cannot be completed, its folder is deleted and every task that names it fails, as it does when a skill fails to install. The run sends orchestrator repackaged skill unavailable once, with the skill, the reason and the framework.
  • A run that installed no pack fails before any task starts, the way it does for a skill the menu cannot supply.
  • install_skill completes such a skill the same way, in both sequences and both harnesses, through the same installer. It picks the pack with the run's match keys and reads it from a scratch folder. A run with no framework gets an error that says so, and nothing is installed.
  • --context-mill-version refuses a release older than 1.66.0, the first that publishes these skills. The message names that release.

The wizard-agent-dag-skills flag, the replaces mapping, the fallback to a replaced skill, the orchestrator_skills tag and pnpm verify:repackaged-skills are gone.

Fixtures

The skill menu and the step and pack pair are recorded from context-mill 1.66.0. pnpm record:context-mill-fixtures now records the pair too: the step skill as published, the pack files that completing it reads, and the completed skill. The recorded completed skill differs from the skill context-mill 1.66.0 publishes for the same step and framework in one line, the name in its frontmatter.

Verified

  • pnpm typecheck and pnpm lint (0 errors).
  • pnpm test: 292 files, 4,450 tests pass, 1 skipped.
  • repackaged-skills.test.ts and repackaged-skill-installer.test.ts cover each way a completion fails: a failed install, a pack that cannot complete the skill, a scan that rejects or fails, a link on the way, and a run with no pack.
  • tool-skill-install.test.ts covers install_skill: a plain skill, a completed one, a run with no framework, and a rejected scan. The pack it reads is gone from disk afterwards.
  • context-mill-version.test.ts covers the floor.

Changes in behaviour to know about

  • A run whose framework pack fails to install now fails before any task starts, when its flow has a skill completed from the pack. It used to continue without the pack.
  • A linear error tracking run that finds no PostHog integration installs integration-install and integration-init, from the menu's integration/install and integration/init categories. install_skill completes both from the framework's pack.
  • install_skill takes an exact menu id, as before. A name that is a skill family, such as integration-v2-install, is not found.
  • FEATURE_FLAGS_UNSUPPORTED still stops feature flags on Kotlin Multiplatform.

Rollout

This is the second of four stacked PRs that remove the old code paths. They merge together and ship in one release.

  • They wait until wizard-agent-dag and wizard-agent-dag-skills have both held at 100% with no stop rule tripped, no run that fetched its prompts, and no run that fell back to an old skill.
  • Both flags stay on in PostHog after the release. Released versions still read them, and a deleted flag reads as off.
  • Cloud moves to the release afterwards, with its context-mill pin at 1.66.0 or later.

🤖 Generated with Claude Code

edwinyjlim and others added 3 commits November 16, 2026 16:54
…om the pack

The bundled prompts name the integration-* step skills. A skill whose menu
entry has frameworkReferences is installed once a run and completed from
the framework's pack. When it cannot be completed, the tasks that name it
fail as they do when a skill fails to install, and a run that installed no
pack fails before any task starts. The wizard-agent-dag-skills flag, the
replaces mapping and the fallback to a replaced skill are removed.

A context-mill pin older than 1.66.0 is refused, since that is the first
release that publishes these skills.

The skill menu and the step and pack pair are recorded from context-mill
1.66.0, and record:context-mill-fixtures records the pair.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
install_skill completes a skill whose menu entry has frameworkReferences
from the framework's pack, through the installer the orchestrator uses,
with the same path and scan checks. The pack is picked with the run's
match keys and read from a scratch folder. A run with no framework gets
an error that says so, and nothing is installed.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A linear error tracking run installs integration-install and
integration-init when PostHog is not integrated yet. install_skill
completes both from the framework's pack.

Comments describe the step skills and the bundled prompts as they are.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@edwinyjlim
edwinyjlim added this pull request to stack #1569 November 17, 2026 01:22
@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci feature-flags
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci feature-flags/django
  • /wizard-ci feature-flags/next-js
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/go-sdk
  • /wizard-ci mcp-analytics/python-sdk
  • /wizard-ci mcp-analytics/ruby-sdk
  • /wizard-ci mcp-analytics/rust-sdk
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

@johncwaters johncwaters left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES

See inline comments.

Automated review by Glimmervoid. Not written by a human

// A folder an earlier run left behind would pass its old pages off as the skill's own.
removeSkillFolder(skillsDir, replacement.id, run.installDir);
const result = await run.install(replacement.id);
removeSkillFolder(skillsDir, skill.id, run.installDir);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[convergent: boundary + redteam] HIGH

install_skill now comes through here with skillsRoot set to .claude/skills, and removeSkillFolder is a plain rmSync(..., { recursive: true, force: true }) with no check for the .posthog-wizard marker. So a project that already has its own .claude/skills/integration-install (hand-installed, or left by a crashed run) loses it, and if completing fails afterwards nothing replaces it. Before this, install_skill went through downloadSkill, which deleted nothing. Could the tool path delete only folders that carry the marker and refuse an unmarked one?

Automated review. Not written by a human

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran it to be sure. In a temp project with my own .claude/skills/integration-install/SKILL.md and no .posthog-wizard marker, I called repackagedSkillInstaller from #1568's head with only the download stubbed to fail, as it would offline:

before: [ 'integration-install' ] marker: false
install rejected: download-failed
after: [] my SKILL.md exists: false

My folder is deleted before the download starts, so a successful install overwrites it too. The fix I'd reach for is to never delete before you have a replacement: install and complete into a wizard temp dir, then swap it in only over a folder that carries the marker, and refuse an unmarked one. That also covers the throw before the try, since a half-finished install never lands in .claude/skills.

Issue 4: the installer owns nothing, deletes anything

Probe
const install = repackagedSkillInstaller({
  installDir: project,
  skillsRoot: '.claude/skills',
  referencePath: 'pack',
  install: async () => ({ kind: 'download-failed', message: 'offline' }),
  scanFile: async () => null,
  onUnavailable: () => {},
});
await install({ id: 'integration-install', frameworkReferences: 'nextjs' } as never);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9d0990e. The skill is now installed, completed and scanned in a temp folder, and only then put in place. It replaces a folder only when it is absent or carries the .posthog-wizard marker, and refuses an unmarked one before anything is downloaded. The orchestrator's own cache is the one root it always replaces. Your repro is a test now: an unmarked folder survives both a failed and a successful install.

const result = await run.install(replacement.id);
removeSkillFolder(skillsDir, skill.id, run.installDir);
const result = await run.install(skill.id);
if (result.kind !== 'ok') throw new Error(result.kind);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[redteam] MEDIUM

This throw sits before the try, so the cleanup below never runs when the install fails. downloadSkill can fail after extracting: a scan engine that throws lands in its catch, which returns success: false and leaves skillDir behind. On the install_skill path that leaves an unscanned step skill, placeholders and all, in .claude/skills/<id>. Moving the install inside the try covers it.

Automated review. Not written by a human

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9d0990e. Nothing is written under .claude/skills until the skill is complete and scanned, and the temp folder is removed in a finally. downloadSkill also removes a folder it created when its install fails.


expect(compareSkillFiles(original, composed)).toEqual([]);
expect(composed['SKILL.md']).not.toBe(original['SKILL.md']);
expect(composed).toEqual(tree(path.join(root, 'completed')));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[code] MEDIUM

The recorder writes completed/ with composeStepSkill (record-context-mill-fixtures.no-jest.ts:103), so this compares composeStepSkill with its own output. If the wizard's fill drifts from context-mill's, recording again regenerates the expected tree and the test stays green. Recording context-mill's published variant as completed/ and comparing with the name line excluded would bring the cross-check back.

Automated review. Not written by a human

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7725d2d. completed/ is now the skill context-mill 1.66.0 publishes for the same step and framework, and the recorder never writes it from the wizard's own fill. The test compares everything but the frontmatter name line.

@sarahxsanders sarahxsanders left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two issues from the local review; details inline.

from Sarah's review bot

// A folder an earlier run left behind would pass its old pages off as the skill's own.
removeSkillFolder(skillsDir, replacement.id, run.installDir);
const result = await run.install(replacement.id);
removeSkillFolder(skillsDir, skill.id, run.installDir);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we preserve unmarked existing skill folders here? installToolSkill now uses this against .claude/skills, so this recursively deletes user-authored files before the download has succeeded. I reproduced this with an unmarked integration-install/my-notes.md: a failed download permanently removes it. Could we stage the completed skill separately and preserve folders we don't own?

from Sarah's review bot

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9d0990e. The completed skill is staged in a temp folder, and a folder without the wizard's marker is refused and left untouched. A test covers an unmarked folder with a failed download.

removeSkillFolder(skillsDir, replacement.id, run.installDir);
const result = await run.install(replacement.id);
removeSkillFolder(skillsDir, skill.id, run.installDir);
const result = await run.install(skill.id);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we include the install in the cleanup block? If extraction or the initial scan throws after writing files, installSkillById returns download-failed and we throw before reaching the cleanup below. I reproduced this with an installer that writes SKILL.md and then returns a failure: the unfinished folder stays on disk. On the tool path, that leaves incomplete or unscanned instructions in .claude/skills.

from Sarah's review bot

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9d0990e. The install now happens in the temp folder inside the try, so a failed or unscanned install never reaches .claude/skills, and the temp folder is always removed.

@gewenyu99 gewenyu99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥸 Reviewed by NotVincent, totally not Vincent. The real Vincent will review it separately. Probably slop, please disregard.

// A folder an earlier run left behind would pass its old pages off as the skill's own.
removeSkillFolder(skillsDir, replacement.id, run.installDir);
const result = await run.install(replacement.id);
removeSkillFolder(skillsDir, skill.id, run.installDir);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this deletes the existing skill folder even when the download then fails, because:

  • removeSkillFolder runs before run.install
  • if install returns anything but ok, we throw, and the old folder is already gone
  • new Error(result.kind) also drops the detail a download-failed result carries

Suggestion: install into a temp folder first and only swap it over the old one once install returns ok, and keep the failure detail in the thrown error.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9d0990e. It stages in a temp folder and only swaps in a finished skill, and the thrown error keeps the detail as download-failed: <message>.

...picked,
frameworkReferences,
}).catch((error: unknown) => {
throw new Error(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if installCompletedSkill fails here (a YARA block or a missing variant), the task gets recorded as no-report instead of the skill error. My line of thought:

  • this rethrows, so the executor treats it as a task failure and retries
  • if the rejected install is cached, the retry gets the same rejection straight back without the original message reaching the report
  • the task then ends with no report, so the real reason is lost

I haven't reproduced this, it's a hunch from reading the code. Worth checking whether the cached rejection surfaces its error.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, and fixed in e880ead. A task whose setup throws is still failed as no-report, but the failure message now carries the thrown reason. The type and the retries are unchanged.

? pickSkillId(FRAMEWORK_PACK_FAMILY)
: undefined;
if (referenceSkillId) {
const ref = await installSkillById(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be good to save the menu somewhere instead of fetching it again here. menuSkillEntries already loaded it, and each installSkillById call fetches skill-menu.json again, which adds up to 22 fetches in one integration run. Maybe fetch it once and resolve upstream.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fsho, will optimize in a followup

skillPaths.push(path.join(completed, 'SKILL.md'));
continue;
}
const result = await installSkillById(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think shared plain skills like posthog-best-practices and integration-mcp get downloaded, unzipped and YARA-scanned again for every task that lists them, and parallel tasks can write into the same folder at the same time. Maybe install each one once (memoize by skill id) and reuse the path.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, and it predates this PR: main installs shared skills once per task as well. Leaving it for the follow-up with the menu caching.

skillId: string,
run: ToolSkillInstall,
): Promise<ToolSkillResult> {
const menu = await fetchSkillMenu(run.skillsBaseUrl);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think each install_skill call fetching the menu and re-downloading the framework pack is a lot of repeated work. Also each call builds its own installer map, so parallel calls don't serialize their writes into .claude/skills. Maybe fetch the menu and pack once per run and share one installer.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on the repeated work. Leaving it for the same follow-up as the menu caching. The write overlap is smaller after 9d0990e, since each call builds in its own temp folder and replaces the target whole.

);

// The pack is read once and thrown away, so it never lands in the project.
const scratch = fs.mkdtempSync(path.join(os.tmpdir(), 'posthog-wizard-'));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these should be caught. fs.mkdtempSync and fs.rmSync are Node's built-in file functions, and both throw instead of returning an error:

  • mkdtempSync throws when the OS temp folder can't be written, for example a full disk (ENOSPC) or no permission (EACCES)
  • rmSync in the finally can throw when a file is still locked, which happens mostly on Windows with antivirus or an open handle (EBUSY, EPERM)

In both cases installToolSkill rejects instead of returning incomplete. Wrapping both in try/catch (and ignoring cleanup errors in finally) would keep the tool result consistent.

Also, deleting the null-menu or failed-packInstall guards still keeps the tool skill tests green, so those paths could use a test.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 0c656ce. The scratch folder is made inside the try and cleanup errors are swallowed, so the call returns incomplete. Tests added for the null menu and the failed pack install.

) as { product: string };
const original = tree(path.join(root, 'original'));

const composed = composeStepSkill(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test nit: the completed/ fixtures are recorded with wizard's own composeStepSkill, so if it drifts from context-mill's builder, re-recording overwrites the expected output and this test still passes. Maybe record completed/ from context-mill's build output instead.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7725d2d. completed/ is recorded from the skill context-mill 1.66.0 publishes, never from the wizard's own fill.

Comment thread src/agent/tools/mcp.ts
import {
incompleteSkillMessage,
installToolSkill,
} from '../runner/sequence/orchestrator/repackaged-skills';

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: importing installToolSkill from the orchestrator means linear install_skill calls send orchestrator repackaged skill unavailable analytics and orchestrator log tags. Maybe pass the event name/tag in, or move the shared part out of orchestrator/.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair. Not moved in this PR. The event keeps its name because the rollout dashboard reads it. Moving the installer out of orchestrator/ can go with the follow-up.

Comment thread src/shared/context-mill-version.ts Outdated
`Invalid context-mill version "${value}". Use a release version such as ${CONTEXT_MILL_FLOOR}, or "latest".`,
);
}
if (belowFloor([Number(match[2]), Number(match[3]), Number(match[4])])) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 1.66.0-rc.1 slips past the floor check, since belowFloor compares only major.minor.patch and ignores the prerelease suffix. Also CONTEXT_MILL_FLOOR is exported but has no callers outside this file.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 70738e2. A prerelease of 1.66.0 is refused, and CONTEXT_MILL_FLOOR is no longer exported.

@edwinyjlim
edwinyjlim marked this pull request as ready for review November 17, 2026 17:09
@edwinyjlim
edwinyjlim requested review from a team as code owners November 17, 2026 17:09
@edwinyjlim
edwinyjlim requested review from JamesPatrickGill, TueHaulund, ablaszkiewicz, fasyy612, hpouillot and ksvat and removed request for a team November 17, 2026 17:09
@posthog-security-review-bot posthog-security-review-bot Bot added security-review Request a security review and removed security-review Request a security review labels Nov 17, 2026
@posthog-security-review-bot

posthog-security-review-bot Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

✅ Security review complete — 3f297bcc953c

1 finding posted as a review, in 7 min.

Add the security-review label to run again on the latest commit.

@posthog-security-review-bot posthog-security-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent-driven security review - findings inline.

@posthog-security-review-bot posthog-security-review-bot Bot removed the security-review Request a security review label Nov 17, 2026
@veria-ai

veria-ai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

Reviews

…s the wizard does not own

A step skill is installed, completed and scanned in a temp folder outside the
project, and put in place only when it is complete. In .claude/skills it
replaces a folder only when that folder carries the wizard's marker; one the
wizard did not install, or a link standing there, is refused before anything
is downloaded. A failed install keeps its detail, and a download that fails
removes the folder it made and keeps one that was there first.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
edwinyjlim and others added 6 commits November 17, 2026 10:12
A menu entry's id names the folder an install writes to. downloadSkill
refuses one that is not a single plain name before it makes a folder or
fetches anything, so no menu can place a skill outside the skills folder.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ublishes

completed/ is the skill context-mill 1.66.0 itself publishes for the pair's
step and framework, read from the skill family the step's menu entry replaces.
The recorder never writes it from the wizard's own fill, and leaves it as it
is when a release no longer publishes that family. The test compares the two
with only the frontmatter name line left out.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A task whose runner throws is failed as no-report once its attempts are spent.
Its failure message now carries the thrown error, so a skill that could not be
installed reads as that and not as an agent that never reported. The type and
the retry behaviour are unchanged.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The scratch folder a framework pack is read from is made inside the try, so a
folder that cannot be made is reported as a skill that could not be completed,
and one that cannot be removed no longer replaces the result of the install. A
pack that fails to install keeps the detail of its failure.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A prerelease of 1.66.0 comes before 1.66.0, so a pin to one is refused like
any older release. The floor is used only in its own file and is no longer
exported.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A completed skill is built in a temp folder and replaces its folder whole, so
nothing calls removeSkillFolder. skillFolder keeps its checks: one plain id,
and a skills root reached through no link.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@edwinyjlim edwinyjlim added the security-review Request a security review label Nov 17, 2026
@posthog-security-review-bot

posthog-security-review-bot Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

✅ Security review complete — 7efd12248354

No findings above the confidence threshold, in 14 min.

Add the security-review label to run again on the latest commit.

@posthog-security-review-bot posthog-security-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent-driven security review - no findings.

@posthog-security-review-bot posthog-security-review-bot Bot removed the security-review Request a security review label Nov 17, 2026

@sarahxsanders sarahxsanders left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like my bot's findings are fixed!!!

@gewenyu99 gewenyu99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fixes from the first round look good to me. Three non-blocking comments inline, none of them need to hold this PR. Thanks!

return { success: true };
} catch (err: any) {
logToFile(`downloadSkill: error: ${err.message}`);
if (!existed) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this cleanup can delete a skill another task just installed, because:

  • parallel tasks install the same plain skill, such as posthog-best-practices, into the same cache folder
  • the first call sees existed === false, the second sees true
  • if the first download fails after the second succeeds, the first removes the folder and the second task gets a SKILL.md path that no longer exists

I reproduced it with two concurrent downloadSkill calls where the first gets a late 404: the second returns success: true and SKILL.md is gone. It needs one of two same-URL downloads to fail, so it is rare. Installing each plain skill once per run, the follow-up you already planned, removes it.

// Only the menu is replaced; `repackaged/` belongs to `verify:repackaged-skills --record`.
fs.mkdirSync(OUT_DIR, { recursive: true });
fs.writeFileSync(path.join(OUT_DIR, 'skill-menu.json'), skillMenu);
// A folder that is not recorded again stays as it is.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: once context-mill#537 ships, a release has no integration-v2-* skill and no replaces, so the recorder keeps completed/ from 1.66.0 and still records step/ and pack/ from the new release. The fixture test then fails as soon as the Flask pack or the init template changes, with no way to record a matching completed/. Maybe leave the whole pair alone when the release publishes no skill for it.

const target = landingFolder(skill, run);
fs.rmSync(target, { recursive: true, force: true });
fs.mkdirSync(path.dirname(target), { recursive: true });
fs.cpSync(staged, target, { recursive: true });

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: if cpSync fails partway, for example on a full disk, the old folder is already removed and a partial one may be left in .claude/skills without the marker. Every later install_skill for that id then refuses it as not the wizard's. Maybe remove target when the copy throws.

@edwinyjlim
edwinyjlim merged commit a5e776a into main Nov 17, 2026
19 checks passed
@edwinyjlim
edwinyjlim deleted the dag/7-repackaged-skills-only branch November 17, 2026 20:43
edwinyjlim added a commit that referenced this pull request Nov 17, 2026
main holds #1565 and #1566 as squash commits, which this branch already
contains commit by commit. The merge takes main's one other change, #1562.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants