Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/shared/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Modules callers reach most:
- `@shared/api`: `Credentials`, `ApiUser`, `ApiProject`.
- `@shared/host-resolution`: `HostResolution`, the immutable snapshot of where the wizard talks to.
- `@shared/fetch-retry`: `fetchWithRetry(url, { fetchImpl?, sleepImpl?, maxAttempts? })`, one retry and failover policy for every critical-path fetch.
- `@shared/skill-menu`: `fetchSkillMenu(skillsBaseUrl, retryOpts?)` returns the parsed `SkillMenu` or `null`; `prepareSkillMenu`, `expandBundleEntry`, `SkillEntry`, `CliEntry`. A fetched menu has each bundle expanded into its variants, and each entry carries its `family`: the key of the category the menu lists it under, with `/` as `-`, so an entry under `integration/install` is in the family `integration-install`.
- `@shared/skill-menu`: `fetchSkillMenu(skillsBaseUrl, retryOpts?)` returns the parsed `SkillMenu` or `null`; `prepareSkillMenu`, `SkillEntry`, `CliEntry`. A fetched menu leaves out an entry that is a bundle of variants and a category that holds only bundles, and each entry carries its `family`: the key of the category the menu lists it under, with `/` as `-`, so an entry under `integration/install` is in the family `integration-install`.
- `@shared/skill-resolution`: `resolveSkill(entries, name, matchKeys)` returns the menu skill a name stands for in one run. A menu id is that skill. Any other name is a skill family, an entry's `family`. A `key:value` tag is a match key: it says which projects a skill is written for, and the run supplies its values as `MatchKeys`. Among a family's skills, one whose match key disagrees with the run's value is dropped, one with no `framework` key is dropped, and the rest rank by keys that agree, then the `default` tag, then menu order. A key the run has no value for never disagrees. A run with no `framework` value resolves no family.
- `@shared/skill-composition`: `completeSkillFromPack(skillDir, packDir, product, boundaryDir)` completes a step skill written once for every framework from that framework's pack. The pack's `variant.json` lists its rules file and each product's pages. Those files are copied into the skill's `references/`, and in `SKILL.md` `{framework_name}` becomes the pack's name while the `{framework_docs}` and `{framework_rules}` lines become the lines that list them. `composeStepSkill(stepFiles, packFiles, product)` is the same on files in memory. Both throw when the pack cannot complete the skill, and both refuse a pack page whose path has a part that is not a plain name or that does not end in `.md`: the install scan reads lowercase extensions and skips dotfiles, so such a page would reach a task unscanned. A `variant.json` with no `rules` reads as one with none. `completeSkillFromPack` checks every target before its first write and throws, writing nothing, when a file would leave `boundaryDir` or pass through a link below it: a folder on the way that is a link, or a file that is one. It does not scan what it writes; the caller scans the `SKILL.md` it rewrites. It takes every file in the skill's folder as the skill's own, so the caller completes a skill in a folder that holds nothing else. `skillFolder(skillsRoot, id, boundaryDir)` is the folder of skill `id` under `skillsRoot`. It refuses an `id` that is not one plain name, since the id comes from the menu, and it reaches `skillsRoot` through no link below `boundaryDir`.
- `@shared/skill-install`: `downloadSkill(entry, installDir, options?)` and
Expand Down
83 changes: 0 additions & 83 deletions src/shared/__tests__/skill-install.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -69,89 +69,6 @@ describe('extractZipArchive', () => {
});
});

describe('extractBundle', () => {
let dest: string;

const bundle = (files: Record<string, string>) => ({
id: 'integration-v2-capture',
variants: { django: files },
});

beforeEach(() => {
dest = fs.mkdtempSync(path.join(os.tmpdir(), 'wizard-bundle-'));
});

afterEach(() => {
cleanup(dest);
});

it('writes only the named variant, including nested paths', () => {
const written = __test.extractBundle(
bundle({ 'SKILL.md': '# skill', 'references/deep/notes.md': 'notes' }),
dest,
'integration-v2-capture-django',
);

expect(written).toBe(2);
expect(fs.readFileSync(path.join(dest, 'SKILL.md'), 'utf8')).toBe(
'# skill',
);
expect(
fs.readFileSync(path.join(dest, 'references/deep/notes.md'), 'utf8'),
).toBe('notes');
});

it('rejects entries that escape the destination', () => {
expect(() =>
__test.extractBundle(
bundle({ '../evil.txt': 'pwned' }),
dest,
'integration-v2-capture-django',
),
).toThrow(/escapes destination/);
expect(fs.existsSync(path.join(dest, '..', 'evil.txt'))).toBe(false);
});

it('rejects absolute entry paths', () => {
expect(() =>
__test.extractBundle(
bundle({ '/etc/evil.txt': 'pwned' }),
dest,
'integration-v2-capture-django',
),
).toThrow(/escapes destination/);
});

it('throws when the bundle lacks the named variant', () => {
expect(() =>
__test.extractBundle(
bundle({ 'SKILL.md': '# skill' }),
dest,
'integration-v2-capture-nextjs',
),
).toThrow(/has no variant/);
});

it('throws a clean error on JSON that is not a bundle', () => {
for (const malformed of [
null,
[],
'oops',
{ id: 'x' },
{ variants: {} },
{ id: 'x', variants: null },
]) {
expect(() =>
__test.extractBundle(
malformed as never,
dest,
'integration-v2-capture-django',
),
).toThrow(/malformed bundle/);
}
});
});

describe('downloadWithRetry', () => {
const url = 'https://example.com/skill.zip';
const noSleep = () => Promise.resolve();
Expand Down
81 changes: 23 additions & 58 deletions src/shared/__tests__/skill-menu.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,84 +46,49 @@ describe('fetchSkillMenu', () => {
expect(result).toBeNull();
});

it('expands a bundle entry into one entry per variant, each with the family of its category', async () => {
const bundled: SkillMenu = {
it('gives each entry the family of its category, and leaves out a bundle of variants with any category it empties', async () => {
const published = {
categories: {
'flow/capture': [
{
id: 'capture',
id: 'flow-capture',
name: 'Capture',
downloadUrl: 'http://localhost:8765/flow-capture.zip',
},
{
id: 'old-capture',
name: 'Capture',
bundle: true,
downloadUrl: 'http://localhost:8765/old-capture.json',
variants: [{ id: 'old-capture-rails' }],
},
],
'old/only': [
{
id: 'old-only',
name: 'Only',
bundle: true,
downloadUrl: 'http://localhost:8765/capture.json',
variants: [
{ id: 'capture-rails', tags: ['framework:rails', 'default'] },
{ id: 'capture-react', tags: ['framework:react'] },
],
downloadUrl: 'http://localhost:8765/old-only.json',
variants: [{ id: 'old-only-rails' }],
},
],
},
};

const result = await fetchSkillMenu('http://localhost:8765', {
fetchImpl: respond(bundled) as any,
fetchImpl: respond(published) as any,
sleepImpl: noSleep,
});

expect(result?.categories['flow/capture']).toEqual([
{
id: 'capture-rails',
tags: ['framework:rails', 'default'],
id: 'flow-capture',
name: 'Capture',
family: 'flow-capture',
bundle: true,
downloadUrl: 'http://localhost:8765/capture.json',
},
{
id: 'capture-react',
tags: ['framework:react'],
name: 'Capture',
family: 'flow-capture',
bundle: true,
downloadUrl: 'http://localhost:8765/capture.json',
downloadUrl: 'http://localhost:8765/flow-capture.zip',
},
]);
});

it("gives each bundle variant the bundle's title and description, keeping its own tags", async () => {
const bundled: SkillMenu = {
categories: {
integration: [
{
id: 'capture',
name: 'Capture',
bundle: true,
title: 'Capture step',
description: 'Instrument the planned events',
downloadUrl: 'http://localhost:8765/capture.json',
variants: [
{
id: 'capture-rails',
tags: ['framework:rails', 'capture'],
},
{ id: 'capture-react', title: 'React' },
],
},
],
},
};

const result = await fetchSkillMenu('http://localhost:8765', {
fetchImpl: respond(bundled) as any,
sleepImpl: noSleep,
});

const [rails, react] = result?.categories.integration ?? [];
expect(rails).toMatchObject({
title: 'Capture step',
description: 'Instrument the planned events',
tags: ['framework:rails', 'capture'],
});
expect(react.title).toBe('React');
expect(Object.keys(result?.categories ?? {})).toEqual(['flow/capture']);
});

it('tags later events with the context-mill release that built the menu', async () => {
Expand Down
46 changes: 1 addition & 45 deletions src/shared/skill-install.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,6 @@ import { fetchWithRetry, type RetryOpts } from '@shared/fetch-retry';
import { PLAIN_NAME } from '@shared/skill-composition';
import { fetchSkillMenu, type SkillEntry } from '@shared/skill-menu';

/** A bundle's files, keyed by variant short id then path. */
export type SkillBundle = {
id: string;
variants: Record<string, Record<string, string>>;
};

/** Extract a zip buffer, refusing entries that escape destDir (zip-slip). */
function extractZipArchive(zip: Uint8Array, destDir: string): number {
const root = path.resolve(destDir);
Expand All @@ -35,37 +29,6 @@ function extractZipArchive(zip: Uint8Array, destDir: string): number {
return written;
}

/** Unpack the one variant this entry names out of a bundle; the rest is noise and never hits disk. */
function extractBundle(
bundle: SkillBundle,
destDir: string,
entryId: string,
): number {
if (
typeof bundle?.id !== 'string' ||
typeof bundle?.variants !== 'object' ||
bundle.variants === null
) {
throw new Error('malformed bundle: expected { id, variants }');
}
const files = bundle.variants[entryId.slice(bundle.id.length + 1)];
if (!files) {
throw new Error(`bundle ${bundle.id} has no variant "${entryId}"`);
}
const root = path.resolve(destDir);
let written = 0;
for (const [entryPath, contents] of Object.entries(files)) {
const target = path.resolve(root, entryPath);
if (target !== root && !target.startsWith(root + path.sep)) {
throw new Error(`bundle entry escapes destination: ${entryPath}`);
}
fs.mkdirSync(path.dirname(target), { recursive: true });
fs.writeFileSync(target, contents);
written++;
}
return written;
}

/** Download a URL to a buffer, retrying transient failures with backoff. */
async function downloadWithRetry(
url: string,
Expand Down Expand Up @@ -120,13 +83,7 @@ export async function downloadSkill(
fs.mkdirSync(skillDir, { recursive: true });
const data = await downloadWithRetry(skillEntry.downloadUrl);
step = 'extract';
const fileCount = skillEntry.bundle
? extractBundle(
JSON.parse(Buffer.from(data).toString('utf8')) as SkillBundle,
skillDir,
skillEntry.id,
)
: extractZipArchive(data, skillDir);
const fileCount = extractZipArchive(data, skillDir);
fs.writeFileSync(path.join(skillDir, WIZARD_MARKER), '');

// Same scan the Bash-install hook runs — TS-path installs (linear
Expand Down Expand Up @@ -320,6 +277,5 @@ export function isSkillInstallCommand(command: string): boolean {

export const __test = {
extractZipArchive,
extractBundle,
downloadWithRetry,
};
35 changes: 5 additions & 30 deletions src/shared/skill-menu.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@ export type SkillEntry = {
downloadUrl: string;
/** The skill family, set on fetch from the category the menu lists the entry under, with `/` as `-`. */
family?: string;
/** This entry's download is a bundle JSON of every variant, not a single skill's zip. */
bundle?: boolean;
/** Human name for the skill. Older menus omit it. */
title?: string;
/** What the skill covers. */
Expand All @@ -25,16 +23,6 @@ export type SkillEntry = {
tags?: string[];
/** The product whose pages, from the framework's pack, complete this skill; see `@shared/skill-composition`. */
frameworkReferences?: string;
/** Menu-only: the variants inside a bundle, expanded into entries of their own on fetch. */
variants?: SkillVariant[];
};

/** One variant inside a bundle entry. */
export type SkillVariant = {
id: string;
title?: string;
description?: string;
tags?: string[];
};

/**
Expand Down Expand Up @@ -64,31 +52,18 @@ export interface SkillMenu {
cliEntries?: CliEntry[];
}

/** Expand a bundle entry into one entry per variant, so the menu reads the same whether a group ships bundled or as zips. */
export function expandBundleEntry(entry: SkillEntry): SkillEntry[] {
if (!entry.bundle || !entry.variants) return [entry];
return entry.variants.map((variant) => ({
...variant,
name: entry.name,
bundle: true,
downloadUrl: entry.downloadUrl,
...(entry.title && !variant.title ? { title: entry.title } : {}),
...(entry.description && !variant.description
? { description: entry.description }
: {}),
}));
}

/** A category's skill family: its key with `/` as `-`, so `integration/install` is `integration-install`. */
const skillFamily = (category: string): string => category.replaceAll('/', '-');

/** The menu as the wizard reads it: each bundle expanded into its variants, and each entry given its family. */
/** The menu as the wizard reads it: each entry given its family, and an entry that is a bundle of variants left out with any category it empties. */
export function prepareSkillMenu(menu: SkillMenu): SkillMenu {
for (const [category, entries] of Object.entries(menu.categories)) {
const family = skillFamily(category);
menu.categories[category] = entries
.flatMap(expandBundleEntry)
const kept = entries
.filter((entry) => !('bundle' in entry && entry.bundle))
.map((entry) => ({ ...entry, family }));
if (kept.length > 0) menu.categories[category] = kept;
else delete menu.categories[category];
}
return menu;
}
Expand Down
16 changes: 0 additions & 16 deletions src/tui/__tests__/skill-entry.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,22 +32,6 @@ describe('resolveSkillEntry', () => {
expect(resolveSkillEntry(MENU, 'nextjs')).toBeNull();
});

it('matches an expanded bundle by its family', () => {
// The menu replaces a bundle with per-framework entries, so no entry keeps
// the bundle id. Every one of them carries the bundle's download URL.
const family = 'integration-v2-error-tracking-step';
const bundled = {
id: `${family}-django`,
family,
bundle: true,
downloadUrl: `https://example.com/${family}.json`,
} as SkillEntry;

expect(resolveSkillEntry([...MENU, bundled], family)?.downloadUrl).toBe(
`https://example.com/${family}.json`,
);
});

it('returns null when nothing matches', () => {
expect(resolveSkillEntry(MENU, 'cobol')).toBeNull();
});
Expand Down
8 changes: 2 additions & 6 deletions src/tui/screens/SkillSourceInfo.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import { CONTEXT_MILL_RELEASES_URL, getSkillsBaseUrl } from '@shared/constants';
* under prefixed ids ('integration-python'); frameworks with variants
* publish several ('integration-nextjs-app-router', '-pages-router').
* Match chain: exact id → `integration-<id>` → unique
* `integration-<id>-*` prefix → bundle family. Ambiguous variants (≥2 prefix
* `integration-<id>-*` prefix. Ambiguous variants (≥2 prefix
* matches) return null — the caller should point at the skills repo instead
* of guessing the wrong variant.
*/
Expand All @@ -43,11 +43,7 @@ export function resolveSkillEntry(
s.id.startsWith(`integration-${skillId}-`),
);
if (variants.length === 1) return variants[0];
if (variants.length > 1) return null;

// The menu expands a bundle into per-framework entries that all carry the
// bundle's family and download URL, so the bundle id matches only by family.
return entries.find((s) => s.bundle && s.family === skillId) ?? null;
return null;
}

export function useSkillEntry(skillId: string | null): {
Expand Down
Loading