Skills: discover user-installed, project, and standalone skills (follow-up to #215)

[jrob5756]

Follow-up to #180 / #215.

#215 ships the skills mechanism and one bundled built-in (conductor). This issue covers the gap found while reviewing it: Conductor discovers no skills from the user's environment, and the registry only accepts a single hardcoded name.

Empirical findings

Probed against a live Copilot SDK session created exactly the way CopilotProvider creates one (no mode, no enable_skills, no enable_config_discovery). Prompt: "List the EXACT names of every skill available to you."

Session config	Skills the agent sees
Conductor today (baseline)	customize-cloud-agent only
+ skills: [conductor] (#215)	conductor, customize-cloud-agent
+ plugin_directories=[<12 enumerated plugin roots>]	all 13 installed + builtin
+ enable_config_discovery=True	everything — project, personal, plugin, builtin

Baseline sees exactly one builtin skill and zero user skills. #215 works as designed, but only for the one skill bundled in the wheel.

Where skills can live

Location	Auto-discovered today	Via enable_config_discovery
~/.copilot/skills/<name>/ (personal)	❌	✅
<repo>/.github/skills/<name>/ (project)	❌	✅
~/.copilot/installed-plugins/<marketplace>/<plugin>/ (plugin)	❌	✅
arbitrary path	❌	via skill_directories

Mechanism details worth recording

• skill_directories accepts either granularity — a single skill dir (containing SKILL.md) or a skills/ root containing several. Both verified working.
• plugin_directories wants the plugin root (the dir containing skills/), and does not recurse. Passing ~/.copilot/installed-plugins alone yields nothing; the */* glob (marketplace/plugin) is required.
• Standalone skills need no plugin and no marketplace — an arbitrary directory works.

Proposed change

Two parts:

1. Let skills: accept paths, not just registered names. Today _BUILTIN_SKILLS is a one-entry dict and get_skill_directory raises SkillNotFoundError for anything else, even though the underlying skill_directories plumbing takes arbitrary paths.

2. Add a discover_skills flag covering all three standard locations.

workflow:
  runtime:
    discover_skills: true          # ~/.copilot/skills + .github/skills + installed plugins
    skills:
      - conductor                            # bundled built-in (ships in the wheel)
      - ./team-skills/acme-widgets           # standalone dir, committed to the repo
      - ~/scratch/skills                     # a skills root — loads everything under it

agents:
  - name: summarizer
    skills: []                     # hermetic — no skills
    discover_skills: false

The ./team-skills/... case is the valuable one for teams: a skill versioned alongside the workflow, with no per-developer install step and no machine-local state.

Implementation notes

• Enumerate the three locations in Conductor and pass skill_directories / plugin_directories explicitly. Do not set enable_config_discovery=True — it reaches the same skill coverage in one flag, but also auto-loads MCP servers from .mcp.json / .vscode/mcp.json in the working directory, silently widening tool access based on which repo the workflow runs in. Same reasoning that made claude_agent_sdk.py set strict_mcp_config=True unconditionally.
• Suggest defaulting discover_skills to false. Ambient discovery makes the same YAML behave differently across machines and CI, which cuts against reproducible runs. One opt-in line is still a long way from enumerating 13 skills by hand.
• Resolve relative paths against the workflow file's directory, consistent with working_dir.
• Path entries need a trust/allowlist decision before landing — reading a SKILL.md from an arbitrary path injects text straight into the agent's context.

Bug: malformed frontmatter fails silently

A SKILL.md whose YAML frontmatter fails to parse is silently skipped — no warning, no error, the skill is simply absent. This cost several rounds of debugging while investigating the above.

The trap is ordinary, and it bit me with a realistic description string:

---
name: acme-widgets
description: Internal ACME widget conventions. Triggers: widget, acme widget.
---

Triggers: inside an unquoted plain scalar makes this invalid YAML. Switching to a block scalar fixed it and the skill loaded immediately:

description: |
  Internal ACME widget conventions. Triggers: widget, acme widget.

Every skill under ~/.copilot/installed-plugins/ uses description: | for exactly this reason.

Proposal: conductor validate should parse each resolved SKILL.md and fail loudly on unparseable YAML or a missing name / description, rather than letting a user point skills: at a directory that quietly never loads.

Related

• Parent: Feature: Conductor Expert — reusable, version-accurate knowledge base for Conductor-aware agents #180
• Implements-on-top-of: feat: Conductor Expert — opt-in knowledge base for Conductor-aware agents (#180) #215
• hermes.py doesn't declare skills= in CAPABILITIES, so it defaults to False and conductor validate rejects skills: on it — yet its execute() docstring describes eager preamble injection working. Eager injection happens in AgentExecutor, upstream of the provider, so hermes could support skills. One of the two should change.

---

Update: this does not generalize across providers

The design above is written in the Copilot SDK's vocabulary (skill_directories, plugin_directories, enable_config_discovery). Three separate things differ per provider: whether a native surface exists, where discovery looks, and what it costs.

Provider matrix

Provider	Native skill surface	Discovery locations	Cost model
copilot	skill_directories, plugin_directories, enable_config_discovery	~/.copilot/skills, .github/skills, ~/.copilot/installed-plugins/*/*	progressive — metadata only
claude-agent-sdk	skills, setting_sources, plugins (see below)	~/.claude/skills, .claude/skills (cwd → repo root), plugin dirs	progressive — ~100 tok/skill
claude (Messages API)	none for local dirs — container beta only	n/a	eager injection
hermes	none	n/a	eager injection
aca	none (host paths unreadable in-sandbox)	n/a	skills=False

For claude.py specifically, the Messages API has no way to point at a local skill directory. Skills there require the code-execution container beta (skills-2025-10-02 + code-execution-2025-08-25 headers) and a container.skills array of skill_ids that must be either Anthropic pre-built (pptx/xlsx/docx/pdf) or previously uploaded to the workspace via the Skills API, capped at 8 per request. So eager injection genuinely is the only local-directory option for that provider — the current design is correct there.

Consequence 1: one flag would mean different things per provider

Copilot discovers from ~/.copilot/…; Claude Code discovers from ~/.claude/…. A single discover_skills: true would therefore surface different skill sets to different agents inside the same workflow, depending on which provider each agent resolves to. That is worse than the machine-portability concern noted above — it is non-determinism within a single run.

Discovery should map onto each provider's own native locations rather than being modeled as one global boolean.

Consequence 2: the cost asymmetry is severe

Measured against a real installed skill set (13 skills):

conductor      109,582 bytes   (7,947 SKILL.md + 101,635 references/)
humanizer       36,097
teams-cache     27,808
code-comments   17,451
conductor-release 11,562
git-workflow     9,950
triage           9,849
web-research     6,265
concept-review   6,095
code-review      6,082
worktree         4,667
gh-cli           4,295
merge-pr         3,470
─────────────────────────────
TOTAL          253,173 bytes  ≈ 63,000 tokens

On the progressive-disclosure providers that is ~1.3K tokens of metadata, with bodies fetched on demand. On the eager-injection providers, AgentExecutor prepends all ~63K tokens to every agent call — roughly a third of a 200K context window, before the agent's own prompt, and paid again on every retry and every validator: call.

Note load_skill_content reads SKILL.md plus the entire references/ tree, which is why the bundled conductor skill alone is 109KB.

Implication: auto-discovery cannot be a single default. Any eager-injection provider needs a size budget with a hard error rather than silently prepending tens of thousands of tokens.

Consequence 3: split resolution from delivery

The cleanest structure given the above:

• Resolution (config → set of skill directories) is provider-agnostic and belongs where the registry lives today.
• Delivery is provider-specific and already exists as the supports_native_skills fork in AgentExecutor.
• Discovery is a third thing, and is provider-specific in location — it should be delegated to each provider rather than resolved centrally into paths.

Related

Split out: #352 covers the claude-agent-sdk native-skills gap and its ambient-loading behavior — a correctness problem on main today, independent of the discovery feature proposed here.

[jrob5756]

Starting to take a look at this.