mcp-tools: no check covers tool-search / deferred tool loading — declared config and server inventory are unowned

[kyle-sexton]

Category: coverage gap (skill scope decision) · Source: agent session, 2026-07-24 — grep-verified against plugins/mcp-tools/, mechanism verified against current official docs.

Filed by an agent session. Every claim below is either a grep result over this repo at the commit in hand, or a verbatim quote from https://code.claude.com/docs/en/mcp.

What mcp-tools:audit covers, and what it disclaims

mcp-tools:audit audits MCP tool-definition quality in source: sixteen criteria (C1–C16 in plugins/mcp-tools/skills/audit/reference/checklist.md) scoring each tool's description, parameters, name, annotations, granularity, and schema self-sufficiency. Its frontmatter draws the boundary explicitly:

Not for: MCP server configuration or connection issues.

Verified: grep -rniE "toolsearch|tool.search|ENABLE_TOOL_SEARCH|defer|token|context cost" plugins/mcp-tools/ returns zero matches (exit 1) across all 13 files in the plugin.

So deferred tool loading is unowned

Tool search is the mechanism that decides whether MCP tool schemas cost context at session start. Per the docs it is on by default — with ENABLE_TOOL_SEARCH unset, all MCP tools are deferred and loaded on demand. It is therefore not an exotic opt-in; it is the ambient behavior, and every deviation from it is a repository-visible declaration.

The plugin whose entire subject is MCP tool quality and tool-selection cost has no check for it. Meanwhile plugins/dometrain/skills/setup/SKILL.md:54 already reasons about ENABLE_TOOL_SEARCH=false ad hoc, inside a consumer plugin — evidence the concern is real and currently homeless rather than absent.

The intent half IS statically auditable

Every one of these lives in a repository file:

Surface	Where	Doc anchor
ENABLE_TOOL_SEARCH — unset | true | auto | auto:N | false	settings env field ("Or set the value in your settings.json env field")	Configure tool search
CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS	settings env	"keeps tool search off, and ENABLE_TOOL_SEARCH can't override it"
permissions.deny: ["ToolSearch"]	settings	"You can also disable the ToolSearch tool specifically"
alwaysLoad: true per server	.mcp.json / plugin-bundled .mcp.json (this repo ships plugins/dometrain/.mcp.json, plugins/miro/.mcp.json)	"Exempt a server from deferral"
"anthropic/alwaysLoad": true in a tool's _meta	the server's own tool definitions	same section
Server instructions field, and the 2 KB truncation on both tool descriptions and server instructions	server source	"For MCP server authors"
ANTHROPIC_BASE_URL pointing at a non-first-party host	settings env	disables tool search by default

Two of these sit squarely inside the plugin's existing lane rather than the disclaimed config lane: _meta's "anthropic/alwaysLoad" is a tool-definition property, and the server instructions field is subject to the same 2 KB truncation that C4 already checks for descriptions — the plugin checks one side of a documented pair and not the other.

Server inventory — how many tool schemas exist to defer at all — is likewise repository data: .mcp.json plus plugin manifests.

The outcome half is not observable statically — name it, don't omit it

A static pass cannot establish that deferral actually happened:

• Tool search requires a supporting model (docs: Claude Sonnet 4.5, Claude Haiku 4.5, Claude Opus 4.5, and later) and is disabled by default on some platforms (Google Cloud's Agent Platform) and behind non-first-party ANTHROPIC_BASE_URL.
• Under auto / auto:N the upfront-vs-deferred split is decided at runtime: "tools load upfront if they fit within 10% of the context window, deferred otherwise" (auto:N for a custom percentage).

Intent is visible; outcome is not. A check that reports a green "tools are deferred" off config alone would be asserting something it cannot see.

Proposal

Add a check to mcp-tools scoped to declared configuration and server inventory, reporting the runtime half as explicitly unobservable rather than silently dropping it — the same honesty the checklist already practises with its SPEC-OPTIONAL / OPINION authority tags.

The decision this needs

Does it widen mcp-tools:audit's declared scope past its own "Not for: MCP server configuration" disclaimer, or land as a sibling action (mcp-tools:context-cost, or similar) so the audit skill's boundary stays intact? Both are defensible; the disclaimer was written deliberately and should be changed deliberately, not eroded by an added criterion.

Mechanism reference (cited, not recapped, so the current text governs): https://code.claude.com/docs/en/mcp#scale-with-mcp-tool-search

[kyle-sexton]

This was generated by AI during triage.

Verified

The coverage gap is confirmed against the repo tree at this commit:

• grep -rniE "toolsearch|ENABLE_TOOL_SEARCH|alwaysLoad" plugins/mcp-tools/ → zero matches. Nothing in the plugin reasons about tool search or deferral.
• plugins/dometrain/skills/setup/SKILL.md:54 does reason about ENABLE_TOOL_SEARCH=false ad hoc — confirming the item's point that the concern is real and currently homeless rather than absent.

The item's own research is unusually well-sourced (every config surface tied to a doc anchor, and the intent/outcome split drawn honestly). Triage did not re-derive the doc claims; they were cited with anchors and the mechanism URL is referenced rather than recapped, so the current text governs.

status: needs-decision cleared — recommendation recorded

The decision this item was parked on is answerable without a human turn, so the parked state is lifted and a recommendation recorded. This is a recommendation, not an applied default — see the routing note below.

RECOMMENDED: land it as a sibling action, not as widened audit scope.

Reasoning, drawn from the item's own framing:

1. The disclaimer ("Not for: MCP server configuration or connection issues") was written deliberately. The item is right that it "should be changed deliberately, not eroded by an added criterion" — and adding a seventeenth checklist criterion is precisely erosion, because it changes the skill's boundary as a side effect of adding a check.
2. A sibling action is additive and reversible. If the boundary later proves artificial, folding a sibling into audit is easy; unpicking a widened scope after consumers depend on it is not. The cheap-to-reverse direction should be the default.
3. The two surfaces the item identifies as already inside audit's existing lane — _meta's "anthropic/alwaysLoad" (a tool-definition property) and the server instructions field (subject to the same 2 KB truncation C4 already checks for descriptions) — are genuinely audit's and can be added there without touching the disclaimer. That split resolves most of the tension: the tool-definition half goes to audit, the config/inventory half goes to the sibling.

Classification and routing

• Work class: C3 scoped, feature-shaped — a new skill action plus checklist additions.
• Disposition: human-gated. Per the admission gate, C3 feature-shaped is human-gated. The recommendation above unblocks the decision so the human's remaining touch is a yes/no rather than a fresh design pass; it does not make the work autonomously dispatchable.
• Priority: medium.

Constraint to preserve whichever option wins

The item's intent-vs-outcome discipline is the most valuable thing in it and is easy to lose in implementation: a static pass cannot establish that deferral actually happened. Model support, platform (GCP Agent Platform), non-first-party ANTHROPIC_BASE_URL, and the runtime auto / auto:N 10%-context split are all unobservable from configuration. Any check shipped here must report the runtime half as explicitly unobservable rather than emitting a green "tools are deferred" it cannot see — the same honesty the existing checklist practises with its SPEC-OPTIONAL / OPINION authority tags.

[kyle-sexton]

Frontier disposition (drain session)

Ruling: Close as not planned for this drain wave (frontier consensus).

Dual-panel drain ruling: advisory, superseded, folded into a program umbrella, or blocked on a host/design carrier without an actionable in-repo slice this session. Re-open with a concrete acceptance brief if a consumer needs the work.

KEEP set for this wave: standing telemetry; host-physical Class C (#631 family); program umbrellas #530/#531/#532/#535; live design carriers #1500/#1553/#1668/#1695/#1716/#1681/#1718; #1360.