Skip to content

Skills: discover user-installed, project, and plugin skills (follow-up to #350) #362

Description

@jrob5756

Split out from #350, which shipped the other four parts (path entries in skills:, loud SKILL.md frontmatter validation, the runtime.skill_injection budget, and hermes declaring skills=True).

This issue covers the one part deliberately deferred: auto-discovering skills already installed in the user's environment, so a workflow can pick them up without enumerating each one.

Why it was deferred

#350's own "Update" section establishes that the original design — a single discover_skills: true flag — does not generalize. Discovery is provider-specific in location:

Provider Discovery locations
copilot ~/.copilot/skills, .github/skills, ~/.copilot/installed-plugins/*/*
claude-agent-sdk ~/.claude/skills, .claude/skills (cwd → repo root), plugin dirs
claude / hermes none — eager injection only
aca none — host paths are unreadable in-sandbox

One boolean would therefore surface different skill sets to different agents inside the same workflow run, depending on which provider each agent resolves to. That is non-determinism within a single run, which is worse than the machine-portability concern the flag was originally weighed against.

What still needs deciding

  1. Shape of the config. Per-provider blocks? A location enum (personal / project / plugins) that each provider maps onto its own paths? Something else? It should not be a single global boolean.
  2. Cost control on eager-injection providers. Skills: discover user-installed, project, and standalone skills (follow-up to #215) #350 added runtime.skill_injection (64KB warn / 128KB error). Measured against a real installed set of 13 skills, discovery would pull in 253,173 bytes ≈ 63,000 tokens — roughly a third of a 200K context window, and it would hard-fail the new budget immediately. Discovery on claude/hermes needs an answer beyond "raise the limit".
  3. Default. Almost certainly false. Ambient discovery makes the same YAML behave differently across machines and CI, cutting against reproducible runs.

Implementation constraint carried over from #350

Do not set enable_config_discovery=True on the Copilot SDK. It reaches the same skill coverage in one flag, but also auto-loads MCP servers from .mcp.json / .vscode/mcp.json in the working directory — silently widening tool access based on which repo the workflow happens to run in. Enumerate the locations and pass skill_directories / plugin_directories explicitly instead. This is the same reasoning behind claude_agent_sdk.py setting strict_mcp_config=True and setting_sources=[] unconditionally.

Empirical notes from #350, still valid:

  • skill_directories accepts either granularity — a single skill dir (containing SKILL.md) or a skills/ root containing several. Both verified working. Conductor now expands roots itself (skills/registry.py::resolve_skills), so discovery can reuse that.
  • plugin_directories wants the plugin root (the dir containing skills/) and does not recurse. Passing ~/.copilot/installed-plugins alone yields nothing; the */* (marketplace/plugin) glob is required.
  • Standalone skills need no plugin and no marketplace — an arbitrary directory works on copilot. Note this is not true on claude-agent-sdk, which has no bare skill-directory surface; Skills: discover user-installed, project, and standalone skills (follow-up to #215) #350 made that a validation error rather than a runtime failure.

Groundwork already in place from #350

  • skills/registry.py::resolve_skills(entries, base_dir) — resolution is provider-agnostic and already handles paths and root expansion.
  • skills/frontmatter.py::read_skill_frontmatter — a discovered skill with broken frontmatter is now reported rather than silently skipped, which matters far more once skills arrive from directories the workflow author never wrote.
  • providers/capabilities.py::uses_native_skills(name) — resolves a provider's delivery mechanism without instantiating it.
  • runtime.skill_injection — the budget any discovery feature will have to answer to.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions