diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 25dbf1271..51ef121d5 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -144,6 +144,12 @@ "source": "./plugins/work-items", "category": "productivity", "tags": ["work-items", "issues", "github", "tracker", "triage", "orchestration", "skill"] + }, + { + "name": "discovery", + "source": "./plugins/discovery", + "category": "development", + "tags": ["discovery", "explore", "research", "codebase", "investigation", "sources", "skill"] } ] } diff --git a/README.md b/README.md index 9fcca763f..9b22e247d 100644 --- a/README.md +++ b/README.md @@ -43,6 +43,7 @@ Browse and manage with `/plugin`. To refresh after updates: `/plugin marketplace | [`firecrawl`](plugins/firecrawl) | Skill | Web scraping, search, crawling, URL discovery, browser interaction, and local file parsing through the `firecrawl-cli` binary — results written to disk and read back selectively to keep large pages out of context, plus a gated maintainer update flow tracking the upstream CLI and skill source. | | [`claude-config-audit`](plugins/claude-config-audit) | Skills | Keeps a repo's Claude Code configuration healthy via three report-first audit skills: `settings-audit` (config files vs upstream truth, with live plugin-drift detection), `memory-health` (CLAUDE.md / rules / auto-memory against official-doc criteria), and `automation-deep-dive` (evidence-gated verdicts on automation gaps). | | [`work-items`](plugins/work-items) | Skill | Manages GitHub Issues as a development work-item tracker: dashboard, taxonomy-labeled creation, a concurrent-safe hold→verify→claim protocol for multi-agent pickup, recurring-schedule checks, TODO scanning, stale-claim auditing, plan decomposition into vertical slices, and structured triage. | +| [`discovery`](plugins/discovery) | Skills | Structured discovery before changes. Ships four skills: `/discovery:explore` and `/discovery:explore-deep` (six-dimension codebase exploration, inline or in a forked subagent) plus `/discovery:research` and `/discovery:research-deep` (three-phase multi-source external research with source tiers, falsification, and recency gates). | Install one: `/plugin install @melodic-software`. diff --git a/plugins/discovery/.claude-plugin/plugin.json b/plugins/discovery/.claude-plugin/plugin.json new file mode 100644 index 000000000..4f42c7d0b --- /dev/null +++ b/plugins/discovery/.claude-plugin/plugin.json @@ -0,0 +1,20 @@ +{ + "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json", + "name": "discovery", + "version": "0.1.0", + "description": "Structured discovery before changes: explore the local codebase (inline or in an isolated forked subagent) and run disciplined multi-source external research with source tiers, falsification, and recency gates — persisting EXPLORE.md / RESEARCH.md handoff artifacts.", + "author": { + "name": "Melodic Software", + "email": "info@melodicsoftware.com" + }, + "license": "MIT", + "keywords": ["discovery", "explore", "research", "codebase", "investigation", "sources", "skill"], + "userConfig": { + "notes_dir": { + "type": "string", + "title": "Notes directory", + "description": "Project-relative directory where discovery artifacts (EXPLORE.md, RESEARCH.md) are written. A working-notes convention declared in your own project's CLAUDE.md or rules takes precedence.", + "default": ".claude/notes" + } + } +} diff --git a/plugins/discovery/README.md b/plugins/discovery/README.md new file mode 100644 index 000000000..efec3fa37 --- /dev/null +++ b/plugins/discovery/README.md @@ -0,0 +1,49 @@ +# discovery + +A Claude Code plugin for **structured discovery before changes** — understand what +IS (the local codebase) and what SHOULD BE (current external sources) before any +code is written. Four skills, two axes: local vs external, inline vs isolated. + +| Skill | Axis | What it does | +|---|---|---| +| `/discovery:explore` | Local, inline | Six-dimension codebase exploration — code reading, git history, project structure, test discovery, build config, environment — plus a `blindspot` mode that surfaces the USER's unknown-unknowns and coaches a better prompt. | +| `/discovery:explore-deep` | Local, isolated | The same explore workflow in a forked subagent: verbose reads and search output stay in the fork; only a short summary returns, with findings persisted to `EXPLORE.md`. Requires `CLAUDE_CODE_FORK_SUBAGENT=1`. | +| `/discovery:research` | External, inline | Three chained research phases (broad → targeted + falsification → preferred sources) with per-claim source tiers, independent-corroborator ratios, a recency gate, and a binary outcome gate before presenting. | +| `/discovery:research-deep` | External, isolated | Dispatcher that routes deep research to the heaviest isolated tier available — a deep-research workflow engine, a forked subagent, or inline as last resort — with a multi-topic check that fans out one agent per separable topic. | + +Both inline skills persist handoff artifacts (`EXPLORE.md` / `RESEARCH.md`) so a +fresh session can resume planning from the artifact alone. + +## Works in any repo + +- **Self-contained.** The research discipline file (source tiers, recency gates, + falsification recipes, failure patterns) and the per-ecosystem discovery + reference ship inside the plugin and are referenced via `${CLAUDE_PLUGIN_ROOT}`. +- **Reads your conventions, assumes none.** Project rules, preferred-source + rosters, per-ecosystem source mappings, and any stated direction come from your + own project's `CLAUDE.md` and rules; where none exist, the skills self-discover + (llms.txt / sitemap probing, canonical-home identification). +- **Graceful degrade.** Adjacent capabilities — a workflow engine, forked + subagents, synthesis MCP servers, documentation agents — are used when present + and substituted when absent; no phase blocks on a missing tool, and substitutions + are documented as gaps rather than silently lowering the bar. + +## Install + +```shell +/plugin marketplace add melodic-software/claude-code-plugins +/plugin install discovery@melodic-software +``` + +## Configuration + +One option, prompted at enable time: + +| Option | Type | Default | Purpose | +|---|---|---|---| +| `notes_dir` | string | `.claude/notes` | Project-relative directory where discovery artifacts (`EXPLORE.md`, `RESEARCH.md`) are written, one subdirectory per topic. A working-notes convention declared in your own project's `CLAUDE.md` or rules takes precedence. | + +## License + +MIT (SPDX-License-Identifier: MIT). See the `LICENSE` file at the root of the +melodic-software/claude-code-plugins repository. diff --git a/plugins/discovery/skills/explore-deep/SKILL.md b/plugins/discovery/skills/explore-deep/SKILL.md new file mode 100644 index 000000000..54e873b06 --- /dev/null +++ b/plugins/discovery/skills/explore-deep/SKILL.md @@ -0,0 +1,57 @@ +--- +name: explore-deep +description: "Run the full explore workflow in an isolated forked subagent so verbose file reads and search output stay out of the main conversation; only a short summary returns, with findings persisted to an EXPLORE.md artifact. Use for thorough or large-scope investigation (10+ file reads or broad search sweeps); requires CLAUDE_CODE_FORK_SUBAGENT=1 — when unset, fall back to inline /explore or a built-in Explore subagent." +argument-hint: "[scope] (e.g., /discovery:explore-deep payments module dependencies, /discovery:explore-deep tests, /discovery:explore-deep git)" +user-invocable: true +disable-model-invocation: false +context: fork +agent: general-purpose +--- + +## Pre-computed context + +Current branch: !`git branch --show-current 2>/dev/null || echo "unknown"` +Working tree status: !`git status --porcelain 2>/dev/null | head -20 || echo "clean"` +Project root: !`git rev-parse --show-toplevel 2>/dev/null || echo "unknown"` + +## Purpose + +You are a forked **general-purpose** subagent running the canonical explore workflow (the sibling `/explore` skill) on behalf of the main session. Your investigation runs in an isolated context — you do NOT see the parent conversation, and the main session does NOT see your file reads, Glob results, or Grep output; only your final summary returns. + +You inherit the parent's full toolset, but this is the **read-only exploration phase**: do NOT Edit source files and do NOT run mutating Bash (no writes/moves/deletes/installs, no git-state changes). The ONLY file you Write is the `EXPLORE.md` artifact in Step 3. Read-only Bash (e.g. `git log`, `git diff`) for the git-history dimension is fine. This read-only boundary is by instruction, not tool-enforced — honor it deliberately. + +This is a forked-execution variant of `/explore`: same investigation discipline, cleaner main-session context. + +## Step 1 — Load the consuming project's conventions + +As a fork you auto-load the project's memory (`CLAUDE.md`), but path-scoped project rules do NOT auto-load in subagent contexts. Before doing scope-relevant work, explicitly Read the consuming project's rule files relevant to `$ARGUMENTS` (its `.claude/rules/` or equivalent — architecture rules, the ecosystem conventions for the file types in scope, testing conventions when scope involves tests). Skip any that don't exist; never invent paths. + +**Scope comes exclusively from `$ARGUMENTS`** — a forked skill does not see the parent conversation, so the caller must pass explicit scope in the invocation. If `$ARGUMENTS` is empty, run a general repository-orientation pass (project structure, build configuration, test layout) and state in both the artifact and your return summary that no scope was provided. + +## Step 2 — Execute the explore workflow + +Follow the sibling `/explore` skill exactly: + +- Cover the relevant subset of its 6 exploration dimensions (codebase reading, git history, project structure, test discovery, configuration, environment) +- Use Glob/Grep/Read aggressively — that's the whole point of running in a fork (the verbose tool calls don't pollute main context) +- Produce its 7-section output report (summary, current state, existing patterns, test coverage, constraints, planned-direction alignment, open questions) + +**Scope**: $ARGUMENTS + +## Step 3 — Persist the artifact + +**Before writing, run the Outcome gate** the `/explore` workflow defines — the binary artifact self-check, not a "did I explore enough?" recap; any FAIL → fix first. + +Write findings to `${user_config.notes_dir}//EXPLORE.md` — derive `` from the exploration scope or current branch name (kebab-case, ≤40 chars). If the consuming project declares its own working-notes convention, that wins over the default location. + +**If EXPLORE.md already exists** there for an unrelated task, write a sidecar `explore-.md` in the same directory instead (kebab-case scope, ≤40 chars) and surface the filename choice in your return summary — the sidecar avoids clobbering prior work. + +## Step 4 — Return summary to main session + +Your conversation history stays in the fork. Return: + +1. A one-paragraph summary (3–5 sentences) of the highest-signal findings +2. The artifact path +3. Any blocking open questions the main session must answer before proceeding + +Do NOT include the full 7-section report in your return — that's what the artifact is for. diff --git a/plugins/discovery/skills/explore/SKILL.md b/plugins/discovery/skills/explore/SKILL.md new file mode 100644 index 000000000..ab6b49bd3 --- /dev/null +++ b/plugins/discovery/skills/explore/SKILL.md @@ -0,0 +1,175 @@ +--- +name: explore +description: "Explore the local codebase before making changes — read code, trace dependencies, scan git history, discover tests, and audit build and tool configuration. Use as step 1 before any code change, for 'what exists for X' investigation, or in blindspot mode to surface the user's unknown-unknowns in unfamiliar territory." +argument-hint: "[scope] (e.g., /discovery:explore payments module dependencies, /discovery:explore tests, /discovery:explore git, /discovery:explore config, /discovery:explore blindspot )" +user-invocable: true +disable-model-invocation: false +--- + +## Pre-computed context + +Current branch: !`git branch --show-current 2>/dev/null || echo "unknown"` +Working tree status: !`git status --porcelain 2>/dev/null | head -20 || echo "clean"` +Project root: !`git rev-parse --show-toplevel 2>/dev/null || echo "unknown"` + +## Routing — context preservation first (three ways to explore) + +Exploration reads many files; keeping that out of the main conversation is what subagents are for. Three ways to run it, by how much context it burns and whether you need this structured workflow: + +- **Built-in Explore subagent — context-preserving default.** For raw "where is X / how does Y work" search, delegate to a fresh Explore subagent ("use a subagent to investigate X"). Fast, read-only, context-isolated. It skips project memory (convention-blind) and does NOT run this 6-dimension workflow or write `EXPLORE.md` — pass key constraints in the prompt when conventions matter. Scale 1→N for coverage: dispatch more Explore subagents (each owning a disjoint area) until nothing relevant to the task is left undiscovered; the main session synthesizes their summaries and persists the artifact. +- **Inline `/explore` — this structured workflow, scoped.** Stay here when ALL hold: ≤~5 known files; tight turn-by-turn iteration; findings feed a same-session edit; you need the `EXPLORE.md` artifact with project rules already loaded. Runs the full 6 dimensions in main context. +- **`/explore-deep` — this structured workflow, forked.** For a single deep pass whose synthesis must ALSO stay off main context: a forked subagent that loads project memory and persists `EXPLORE.md` itself before returning only a summary. Pass explicit scope in the invocation arguments — a fork does not see the parent conversation. Requires `CLAUDE_CODE_FORK_SUBAGENT=1`; if unset, fall back to the built-in Explore subagent or inline. + +**Coverage discipline** when fanning out: (1) write a numbered gap-list before any deepen pass; (2) fan out by disjoint area — never split the six dimensions across agents; (3) the main session synthesizes and writes `EXPLORE.md` (built-in Explore agents cannot write it). + +## Purpose + +Exploration is the prerequisite for everything — you cannot change what you do not understand. Goal: **maximum local knowledge** before any action. Skipping exploration leads to wrong assumptions, missed patterns, broken conventions, and rework. + +Local counterpart to `/research` (external sources). Together: `/explore` for what IS, `/research` for what SHOULD BE. + +**Philosophy**: invest in understanding before acting. Reading 20 files takes seconds; fixing a wrong assumption takes minutes to hours. When in doubt, read more code. + +**Plan-mode for high-risk exploration (optional)**: when exploring unfamiliar code in a high-blast-radius area (security boundaries, critical infrastructure, code you might accidentally modify mid-investigation), switch into plan mode for harness-level read-only protection. Routine exploration of well-understood code does not need this. + +## Scope + +Explore the following: $ARGUMENTS + +If no specific scope was provided above, infer the exploration scope from the current conversation context — identify what area of the codebase is relevant to the task at hand and explore that. + +## Exploration dimensions + +Cover the relevant subset of these dimensions. Not all apply to every task — use judgment about which matter for the current scope. + +### 1. Codebase reading + +Read the actual code before forming opinions about it. + +- **Targeted files** — Read files directly relevant to the task. Use Glob by pattern, Grep by content +- **Adjacent code** — Read code that calls, is called by, or is structurally similar to the target. Understand the neighborhood, not just the target +- **Existing patterns** — Before proposing a new pattern, search for how the same concern is handled elsewhere in the repo. Reuse > reinvent (unless the existing pattern is an anti-pattern or well outside modern best practices and not documented as a pragmatic decision) +- **Convention files** — Check the consuming project's own `CLAUDE.md` and project rules for conventions that constrain the solution space +- **Reference source as spec** — when the task points at an existing implementation to match (a vendored library, another module, even another language), READ that source as the authoritative spec + +### 2. Git history + +Code has context only git reveals — who changed it, when, why, and what else changed alongside it. + +- `git log --oneline -20 ` — recent change frequency and commit style +- `git log --oneline --all --since="2 weeks ago"` — recent repo-wide activity +- `git diff HEAD~5 -- ` — what changed recently in the target area +- `git blame ` — when specific lines were last touched and by whom +- **Missing files** — when `git status` or history references files that don't exist on disk, **ask the user before investigating** via git archaeology. They may be intentionally deleted + +### 3. Project structure + +Understand how the pieces fit together before moving any of them. + +- **Directory layout** — if the project documents its repository structure, verify the doc matches reality; otherwise map the tree yourself +- **Project references / imports** — map the dependency graph by grepping the ecosystem's import/reference token across its build-config files (per-ecosystem tokens: `${CLAUDE_PLUGIN_ROOT}/skills/explore/reference/ecosystem-discovery.md`) +- **Solution / workspace membership** — check the repo's solution or workspace file at the root for what's included +- **Layer boundaries** — respect any dependency-direction rules the project declares +- **Planned direction** — cross-reference findings with any stated direction in the project's own `CLAUDE.md` or docs. Assess how changes must fit the repo's current state AND planned direction + +### 4. Test discovery + +Tests are executable documentation. They reveal intended behavior, edge cases, and existing coverage. + +- **Find test projects** — Glob the per-ecosystem test patterns in `${CLAUDE_PLUGIN_ROOT}/skills/explore/reference/ecosystem-discovery.md` +- **Co-located tests** — check whether unit tests live next to their source (sibling test project, `__tests__/`, adjacent `_test.go`) or in a separate tree +- **Cross-cutting tests** — a repo-root `tests/` for architecture, dependency, or naming-rule tests that span multiple libraries +- **Test patterns** — read existing tests (start with 2-3, scale to the number of distinct patterns in play) to understand naming conventions, assertion style, and fixture patterns before writing new ones +- **Coverage gaps** — identify areas with no test coverage that the current task touches + +### 5. Configuration and build state + +Build configuration constrains what's possible. Understand it before fighting it. + +- **Build configs** — read the ecosystem's build / package / config files (per-ecosystem lists in `${CLAUDE_PLUGIN_ROOT}/skills/explore/reference/ecosystem-discovery.md`) +- **Analyzer / lint config** — `.editorconfig` for shared severity levels; ecosystem-specific analyzer/linter files +- **Package versions** — check the ecosystem's manifest (lockfile + central-version-management file if applicable) +- **CI/CD** — `.github/workflows/` (or the project's CI equivalent) for what's validated on every PR + +### 6. Environment and machine state + +When the task involves tooling, MCP servers, or infrastructure: + +- **Installed versions** — run the per-ecosystem version commands in `${CLAUDE_PLUGIN_ROOT}/skills/explore/reference/ecosystem-discovery.md` +- **MCP server status** — test with a read-only call before depending on it +- **Worktree state** — `git worktree list`, current branch, uncommitted changes +- **Local config** — project-local settings for env vars and tokens (don't read secrets, just verify presence) + +## Exploration modes + +The `$ARGUMENTS` value shapes the exploration focus: + +| Argument | Focus | Key actions | +|----------|-------|-------------| +| *(empty)* | Infer from conversation context | Read relevant code, check git history, verify tests exist | +| `` (a module, namespace, or directory) | Targeted area deep-dive | Read all files in area, trace dependencies in and out, find tests | +| `deps` or `dependencies` | Dependency graph analysis | Map project references, check for circular deps, verify layer rules | +| `tests` | Test structure and coverage | Find all test projects, read test patterns, identify gaps | +| `git` | Recent change history | `git log`, active branches, recent contributors, change velocity | +| `config` | Build and tool configuration | Read `.editorconfig`, build configs, analyzer settings, CI workflows | +| `` | Single file deep-dive | Read file, its tests, its callers, its git history | +| `blindspot ` | The USER's unknown-unknowns, not the agent's | See "Blindspot mode" below | + +Multiple arguments combine: `/discovery:explore payments deps tests` explores that area's dependencies AND test coverage. + +## Blindspot mode + +Every other mode builds the AGENT's local knowledge; blindspot mode builds the USER's. Run it when the user is about to work in territory they don't know — an unfamiliar codebase area OR an unfamiliar domain vocabulary — and the goal is a better prompt. + +1. **Intake** — ask the user's starting point first (one question). Blindspot output calibrates to that disclosure. +2. **Scan** — codebase lane: read the target area (dimensions 1-3 above) looking specifically for things the user's framing didn't account for — existing patterns they'd duplicate, constraints they'd violate, historical decisions they'd re-litigate, adjacent code their change would break. Domain lane: build a lightweight vocabulary ladder grounded in sources fetched this session (repo files, official docs) — never bare training recall. +3. **Output — blindspot cards.** One card per blindspot: the gap, why it matters here, and a copyable prompt-fix line. Close by assembling the fixes into ONE improved implementation prompt the user can run next. +4. **Escalate when depth warranted** — a domain too deep for a lightweight ladder gets a recommendation to run proper external research (`/research`) or whatever structured-learning capability the environment provides. + +Blindspot mode does NOT write EXPLORE.md by default — its deliverable is the user's understanding plus the improved prompt. Offer the persist only when findings double as stage-1 exploration. + +## Output format + +Present exploration findings as: + +1. **Summary** — 2-3 sentence overview of what was found +2. **Current state** — key facts about the explored area (structure, patterns, dependencies). When the explored module has a domain-vocabulary or glossary file, frame findings using the module's domain vocabulary +3. **Existing patterns** — how similar concerns are handled elsewhere in the repo +4. **Test coverage** — what's tested, what's not, what test patterns are used +5. **Constraints** — analyzers, conventions, layer rules, or CI gates that constrain the solution +6. **Planned direction alignment** — how findings relate to any direction the project documents +7. **Open questions** — anything that needs clarification before proceeding. **Surface these to the USER**, each with a one-line recommended default + escape hatch. Silent downstream resolution of surfaced open questions is an anti-pattern + +If invoked standalone, present findings directly. If invoked as part of a larger workflow, findings feed into subsequent research and planning steps. + +## Outcome gate (before EXPLORE.md handoff) + +Blindspot-only runs SKIP this gate — their deliverable is blindspot cards plus an improved prompt, not the 7-section artifact (run it only when the user opts into the EXPLORE.md persist). For all other modes: before writing EXPLORE.md (or returning the summary), check the artifact against **binary criteria read off it** — not a "did I explore enough?" recap. Any FAIL → return to the named dimension and fix before handoff: + +- **Every Output-format section populated with specifics** — each of the 7 sections carries concrete findings, not placeholders or "TBD". +- **Every load-bearing area covered OR listed as a numbered gap** — nothing the task plausibly depends on is silently unexplored. +- **Conclusion-driving claims are Read-verified, not inferred from a filename or grep hit** — anything a downstream decision rests on came from reading the file or code. +- **Open questions surfaced to the user**, each with a recommended default. + +## Final step: persist artifact for handoff + +Blindspot-only runs SKIP this step (see "Blindspot mode"). For all other modes: write the exploration output to `${user_config.notes_dir}//EXPLORE.md` — derive `` from the exploration scope or current branch name (kebab-case, ≤40 chars). If the consuming project declares its own working-notes convention (in its `CLAUDE.md` or rules), that convention wins over the default location. This file is the authoritative stage summary — a fresh session must be able to resume external research or planning reading only this artifact. + +The artifact's Findings section follows the 7-point Output format above, and a closing Next-stage-handoff names what external research (`/research`) or planning needs. + +If exploration spans many sub-areas and EXPLORE.md exceeds ~2000 words, split overflow into sibling `explore-.md` files in the same directory and keep EXPLORE.md as the index. + +## Gotchas + +- **Fan-out without a numbered gap-list** — dispatching subagents before writing gaps produces duplicate reads and missed areas. The gap-list is the coverage-discipline gate. +- **Handing off with placeholder sections** — every Output-format section needs specifics or an explicit numbered gap. "TBD" fails the outcome gate. +- **Inferring from filenames without Read** — grep hits are discovery only; conclusion-driving claims need Read verification. +- **Investigating deleted files without asking** — when `git status` shows intentional deletes, ask before archaeology. + +## What this skill does NOT do + +- **Does not research externally** — that's `/research`. This skill reads local code, git, and file system only. Sole carve-out: the blindspot domain lane may fetch official docs to ground its vocabulary ladder +- **Does not make changes** — it explores. Execution is a separate step +- **Does not make decisions** — it presents what IS. The planning step decides what SHOULD BE +- **Does not skip dimensions for "simple" tasks** — a quick bug fix still benefits from reading the surrounding code and checking for tests +- **Does not substitute for reading** — when uncertain, Read the file. Don't infer from file names or git log alone diff --git a/plugins/discovery/skills/explore/reference/ecosystem-discovery.md b/plugins/discovery/skills/explore/reference/ecosystem-discovery.md new file mode 100644 index 000000000..cc0bbef35 --- /dev/null +++ b/plugins/discovery/skills/explore/reference/ecosystem-discovery.md @@ -0,0 +1,93 @@ +# Ecosystem discovery — explore + +Per-ecosystem discovery primitives consumed by the explore skill's Dimensions 3–6 +(project structure / test discovery / configuration & build state / environment). + +Sub-keys: + +- `test-globs` — glob patterns identifying test projects / files (Dimension 4) +- `test-content-grep` — content regex for ecosystems that keep tests inside + source files, where no glob can find them (Dimension 4; only where needed) +- `build-configs` — build / package / config files to read when exploring + "Configuration and build state" (Dimension 5) +- `dependency-grep` — content regex grepped across source / project files to map + the dependency graph (Dimension 3) +- `runtime-version-cmd` — command to check the installed runtime version (Dimension 6) + +Use only the ecosystems the consuming repo actually contains. Where the consuming +project's own conventions differ (a custom test layout, a nonstandard workspace +file), the project's conventions win — this table is the generic starting point. + +```yaml +ecosystems: + dotnet: + test-globs: + - "**/*Tests*/*.csproj" + - "**/*.Tests/*.csproj" + build-configs: + - "Directory.Build.props" + - "Directory.Build.targets" + - "Directory.Packages.props" + - "global.json" + - "*.sln" + - "*.slnx" + - "*.csproj" + dependency-grep: "ProjectReference" + runtime-version-cmd: "dotnet --version" + + typescript: + test-globs: + - "**/*.test.{ts,tsx,js,jsx}" + - "**/*.spec.{ts,tsx,js,jsx}" + - "**/__tests__/**" + build-configs: + - "package.json" + - "tsconfig.json" + - "pnpm-workspace.yaml" + - ".nvmrc" + dependency-grep: "\"dependencies\"|import .* from" + runtime-version-cmd: "node --version" + + python: + test-globs: + - "**/test_*.py" + - "**/tests/*.py" + build-configs: + - "pyproject.toml" + - "requirements*.txt" + - ".python-version" + dependency-grep: "^import |^from " + runtime-version-cmd: "python --version" + + go: + test-globs: + - "**/*_test.go" + build-configs: + - "go.mod" + - "go.work" + # Go imports span multi-line import ( ... ) blocks; the second alternative + # matches the quoted module paths inside them. + dependency-grep: "^import |^\\s*\"[A-Za-z0-9._~/-]+\"$" + runtime-version-cmd: "go version" + + rust: + test-globs: + - "**/tests/*.rs" + test-content-grep: "#\\[cfg\\(test\\)\\]" + build-configs: + - "Cargo.toml" + - "Cargo.lock" + dependency-grep: "^use |\\[dependencies\\]" + runtime-version-cmd: "rustc --version" + + java: + test-globs: + - "**/src/test/java/**/*.java" + build-configs: + - "pom.xml" + - "build.gradle" + - "build.gradle.kts" + - "settings.gradle*" + dependency-grep: "|implementation\\(" + runtime-version-cmd: "java --version" +``` diff --git a/plugins/discovery/skills/research-deep/SKILL.md b/plugins/discovery/skills/research-deep/SKILL.md new file mode 100644 index 000000000..cb75804a6 --- /dev/null +++ b/plugins/discovery/skills/research-deep/SKILL.md @@ -0,0 +1,86 @@ +--- +name: research-deep +description: "Dispatch deep external research to the heaviest isolated execution tier available — a workflow engine, a forked subagent, or inline as last resort — keeping the main conversation clean while the full research discipline runs. Use for broad, multi-source, comparison, or migration research; for a single small lookup use the research skill directly." +argument-hint: "[topic] (e.g., /discovery:research-deep best practices, /discovery:research-deep migration guide)" +user-invocable: true +disable-model-invocation: false +--- + +## Pre-computed context + +Current branch: !`git branch --show-current 2>/dev/null || echo "unknown"` + +## Purpose + +`/research-deep` is the **dispatcher** for deep external research — a depth/execution variant of the sibling `/research` skill. Same research contract (3-phase discipline, source-tier ratio, recency gate, mandatory falsification, cited `RESEARCH.md` artifact); heavier execution that keeps the main session's context clean. It selects ONE execution tier from tool availability + task heaviness, then surfaces the same summary contract regardless of tier. + +This skill runs **inline (main context)** — it dispatches; the chosen tier provides the context isolation. It must run in main context to reach the Workflow tool when one is available (a subagent cannot dispatch workflows). + +## Topic + +$ARGUMENTS + +If no topic was provided, infer it from the current conversation — identify the technical claim, decision, or implementation being worked on and research that. + +## Dispatch decision (multi-topic check, then three tiers) + +**Multi-topic check — run FIRST, before any tier.** Count the independent sub-topics in the ask (numbered list, enumerated questions, separable subjects that share no claims). **N ≥ 2 separable topics → do NOT dispatch an engine on the combined blob.** An engine decomposes ONE question into generic research *angles*; fed a multi-topic blob, every broad agent researches all N topics shallowly — N× the wall-clock and tokens for worse depth. Instead: spawn **N parallel topic agents** (Agent tool, `general-purpose`, one per topic, each running the full `/research` discipline; instruct each to cite primary sources by URL — a subagent return without citations is ungrounded synthesis). Each agent writes its own artifact as a sibling `research-.md`; the main session then synthesizes `RESEARCH.md` from the per-topic artifacts. An engine is for a SINGLE contested or deep question that needs falsification rounds and adversarial claim-checking. + +For a single-topic ask, detection is **engine-biased**: prefer the heaviest available tier UNLESS the task is clearly small/targeted. Unknown scope or any doubt → heavier tier. + +| Tier | Condition | Execution | +|---|---|---| +| 1 — workflow engine (preferred) | The Workflow tool is available AND a deep-research workflow exists (a built-in deep-research workflow, or one the consuming project ships) AND the task is heavy/broad (or unknown scope) | Dispatch that workflow with the topic | +| 2 — forked subagent | No workflow path AND the task is heavy | Spawn an isolated `general-purpose` agent running the full `/research` discipline | +| 3 — inline | Task clearly small/targeted (single fact, one obvious source, narrow lookup) | Run `/research` inline in this session | + +- **Heavy/broad** = multi-source, multi-vendor, comparison/migration, unfamiliar domain, or research that would flood main context with 9+ external queries. +- **Clearly small** = a single verifiable fact from one obvious source. Even here the full `/research` discipline applies — task size never reduces depth. +- **Multi-topic parallel agents** = each topic agent still runs the FULL `/research` discipline (3 phases, source tiers, falsification) — the split changes orchestration, never depth. + +### Tier 1 — workflow engine (preferred) + +If your tool list includes the Workflow tool and a deep-research workflow is available (check the consuming project's workflow registry first — a project-provided engine may superset the built-in one), dispatch it with the topic and, if it accepts one, the artifact destination (`${user_config.notes_dir}//RESEARCH.md`). The engine runs in the background; its completion notification carries the summary + artifact path. Surface those to the user. Do **NOT** re-run the research inline. + +If no workflow engine resolves, fall through to Tier 2. + +### Tier 2 — forked subagent fallback + +Spawn a subagent that runs the canonical `/research` workflow in an isolated context: + +```text +Agent({ + subagent_type: "general-purpose", + description: "Deep research (isolated)", + prompt: "Run the discovery plugin's research skill (/discovery:research) on: and + follow its disciplines exactly — the skill loads its own discipline file; do NOT + reconstruct it here. Carry-verbatim reminders: queries SCALE to open questions, + not a flat floor; primary source fetched DIRECTLY, not via the SERP; mandatory + Phase-2 falsification. Discover the research tools connected this session — don't + assume a fixed set. Every accepted claim needs a primary source cited by URL + captured this run; uncited claims are ungrounded synthesis. RUN THE OUTCOME GATE + before returning. Write the RESEARCH.md artifact per the skill's Final step. + Return ONLY a one-paragraph summary + the artifact path + any unresolved questions." +}) +``` + +`general-purpose` (not a read-only Explore agent) because Phase 3 needs MCP/tool access and the artifact must be written. + +### Tier 3 — inline (clearly small task) + +Run `/research` inline in this session. No subagent, no workflow. The full `/research` discipline still applies. + +## Relationship to `/research` (parent skill) + +This variant tracks `/research`'s conventions — same discipline file, same artifact contract, same outcome gate. There is no separate copy here; update the parent and this dispatcher follows. + +## What this skill does NOT do + +- Does NOT make decisions or write code — research only; the planning step (or user) decides. +- Does NOT skip phases for "simple" topics — task size does not reduce depth. +- Does NOT run the deep pass itself in main context — it dispatches; Tier 1 (engine) or Tier 2 (subagent) provides the context isolation. + +## See also + +- `/research` — the canonical 3-phase workflow (Tiers 2 + 3 run it; a Tier-1 engine supersets it) +- `${CLAUDE_PLUGIN_ROOT}/skills/research/context/discipline.md` — the shared discipline file diff --git a/plugins/discovery/skills/research/SKILL.md b/plugins/discovery/skills/research/SKILL.md new file mode 100644 index 000000000..e9409a474 --- /dev/null +++ b/plugins/discovery/skills/research/SKILL.md @@ -0,0 +1,184 @@ +--- +name: research +description: "Multi-source external research in three chained phases — broad, targeted + falsification, preferred sources — with per-claim source tiers, recency checks, and a binary outcome gate before presenting. Use to verify a technical claim, evaluate libraries or approaches, compare X vs Y, or ground any decision in current authoritative sources instead of training data." +argument-hint: "[topic] (e.g., /discovery:research best practices, /discovery:research hook event schema, /discovery:research query optimization)" +user-invocable: true +disable-model-invocation: false +--- + +## Pre-computed context + +Current branch: !`git branch --show-current 2>/dev/null || echo "unknown"` + +## Purpose + +External research is mandatory before acting on external facts. Goal: **maximum knowledge, maximum consensus, latest information** from authoritative + official sources. Training data drifts; library APIs change; SEO content farms outrank authoritative sources; AI synthesis tools repackage the same secondary blogs as "multi-source." Cross-tool consensus + primary-source priority + recency verification drive accuracy. + +Local counterpart: `/explore` (what IS in the repo). This skill covers what SHOULD BE, per current external sources. For context-heavy passes, use `/research-deep` (dispatches to an isolated execution tier; keeps main context clean). + +**Philosophy**: more tokens + more time = more accuracy + less rework. Insufficient research is the #1 source of rework. Deploy a **research team**, not a single lookup. Every invocation gets full depth regardless of task size. + +## Topic + +Research the following topic: $ARGUMENTS + +If no specific topic was provided above, infer the research topic from the current conversation context — identify the technical claim, decision, or implementation being worked on and research that. + +## Mandatory disciplines (non-negotiable) + +Full recipes and rationale: `${CLAUDE_PLUGIN_ROOT}/skills/research/context/discipline.md` (also the canonical source-tier table for this plugin). + +1. **3 phases minimum** — Phase 1 (broad), Phase 2 (targeted, informed by Phase 1, includes falsification), Phase 3 (preferred-sources / tool-ecosystem fallback) +2. **Queries scale to open questions — the floor is a starting point, not a target** — Phase 1 opens with ≥3 queries to seed the evidence base; Phase 2 and Phase 3 each run **one query per unresolved gap/conflict** surfaced by the prior phase's written analysis (≥3, no upper cap). Stopping at the floor while gaps remain is a violation — read every floor below as "at least," never "exactly" +3. **3 distinct tool types minimum per phase** — using only one search engine + one synthesis tool for a phase is a violation; mix in direct fetches, doc-MCP servers, `gh api`, or documentation agents your environment provides +4. **4+ distinct tool types across the topic** — phases cannot share the same 3 tools end-to-end. Cross-phase tool diversity is the consensus-driving mechanism +5. **Source-tier ratio per claim** — every accepted claim has ≥1 Tier 0/1 (primary source captured this turn) PLUS ≥2 independent corroborators, REGARDLESS of how authoritative the primary is: a canonical doc does not waive corroboration (it can be stale). Three synthesis-tool citations of three blogs = 1 Tier 2 source, NOT 3. Track diversity per claim +6. **Recency gate — first-party docs lag releases** — one query MUST fetch the LATEST upstream changelog or release notes this turn and confirm the claims are current as of it. The 30/14/90-day windows (standard / very active / conceptual) bound how stale a cited doc may be before it needs that cross-check — a stable project whose latest release is older than the window still passes once that release is confirmed current. Major version bump = invalidate prior docs INCLUDING first-party: treat any doc-vs-changelog lag as a conflict to resolve, not a closed answer +7. **One falsification query in Phase 2 (MANDATORY)** — Phase 2 must include exactly one query that attempts to FALSIFY the leading hypothesis from Phase 1 +8. **Broad-topic auto-detect → doubled minimums** — when the topic involves 2+ vendors / 2+ tools / 3+ proper-noun products / comparison ("X vs Y") / migration ("X replaces Y") → 6+ queries per phase, 12+ total, 5+ tool types, 4+ Tier 0/1 sources per claim +9. **Phases chain through a WRITTEN analysis** — Phase 2 consumes the gap/conflict/leading-hypothesis list emitted at the end of Phase 1; each Phase 2 query maps to a named entry in it. Phase 3 chains the same way off the Phase 1+2 list. A query not traceable to a prior-phase gap is unchained — the written list IS the broad→deep link, intent is not +10. **Task size does NOT reduce phase count** — a one-line config change gets the same treatment as a multi-file feature +11. **Confidence tracked per claim** — HIGH / MEDIUM / LOW per the discipline file's "Confidence calibration." Do NOT accept LOW-confidence claims as a basis for code edits — iterate until HIGH +12. **Primary source fetched directly, not via the SERP** — for every accepted claim, name the canonical doc home and fetch it directly with whatever direct-fetch tool is connected this session; SERP + synthesis tools only DISCOVER what to fetch and find corroborators, never serve as the terminal source +13. **Outcome gate before presenting (MANDATORY)** — the run self-checks its own evidence table + written gap lists + fetch log against binary criteria; any FAIL returns to the named phase (see "Outcome gate") + +## Phase 1: Broad Research (3+ queries, 3+ tool types) + +Cast a wide net. Objective: establish the initial evidence base and identify what we don't know yet — survey the landscape before spending depth on any single source. + +### Discover your tools first — don't assume a fixed set + +Research tools vary by session: MCP servers connect and disconnect, CLIs come and go. Before launching, take stock of what's actually available THIS session — your active/deferred tool list, the MCP server-instruction blocks already injected into context, and the project's MCP registry. Map the source CATEGORIES below to whatever you have; never hard-depend on one server (a docs-MCP server absent → WebFetch the docs site directly instead). + +**Launch ≥3 queries across ≥3 source categories in parallel:** + +| Source category | What it gives you | Reach for whatever's connected | +|---|---|---| +| **Official docs** | The authoritative primary for an ecosystem/library | the ecosystem's canonical docs site, fetched directly. If the consuming project ships a per-ecosystem source mapping (check its `CLAUDE.md`/rules), use it; else identify the canonical home yourself. When the topic centers on a specific library/site, probe its `llms.txt` / sitemap first to enumerate the doc set | +| **Upstream source + releases** | Ground truth + recency for a tool/library | the GitHub repo, releases, `CHANGELOG.md` — required for the recency gate | +| **Package registry** | Versions, dependencies, publish dates | the ecosystem's registry (NuGet / PyPI / npm / crates.io / Maven Central) | +| **Spec / standard** | Definitive behavior for a protocol/language | the RFC, language spec, or standard document | +| **AI-synthesis — DISCOVERY ONLY** | Fast breadth + citations to chase | a synthesis tool to FIND primaries and corroborators — never the terminal source for a claim. When it exposes a depth/quality knob, max it (accuracy over speed) | +| **Community corroborators** | Independent agreement / dissent | named-author blogs, top-voted Q&A, practitioner posts — corroborators, not primaries | + +- **Prefer direct-context web** (WebSearch / WebFetch in the main session) for the highest-value queries — results land without summarization loss. +- **Vendor-tool topics** — when the topic is the AI coding tool itself (or any fast-moving vendor tool), prefer a dedicated documentation agent/skill if your environment provides one; general synthesis tools carry stale info for fast-moving tools. Also check the upstream issue tracker for known bugs in the version in use. + +### Phase 1 output — write this list before composing any Phase 2 query + +**STOP. Emit a written analysis block** — this IS the broad→deep chaining mechanism. Phase 2 queries are composed FROM it, not alongside it. The block MUST contain: + +- **Leading hypothesis** — what the evidence points toward +- **Gaps** (numbered) — each claim not yet backed by ≥1 primary (Tier 0/1) + 2 independent corroborators, plus any open question. Every numbered gap earns a Phase 2 query — the gap count sets the Phase 2 query count +- **Conflicts** (numbered) — disagreements between sources; each earns a resolving Phase 2 query +- **Tool-diversity audit** — distinct tool types used; if <3, this phase failed — re-run before proceeding +- **Recency status** — upstream changelog/release fetched? If not, queue for Phase 2 +- **Falsification candidate** — the most load-bearing claim that, if wrong, invalidates the rest. That's the Phase 2 falsification target + +Phase 2 is not "launch 3 queries" — it is "close every numbered gap + conflict above, plus the one falsification query." If that totals 6, run 6. + +## Phase 2: Targeted + Falsification (one query per Phase 1 gap/conflict + 1 mandatory falsification) + +Objective: fill gaps, resolve conflicts, strengthen low-confidence claims, AND attempt to break the leading hypothesis. + +**One query MUST be a falsification attempt** against the Phase 1 leading hypothesis. See the discipline file's "Falsification step" for query patterns. Without this step, Phase 2 is confirmation bias by default. + +**Remaining queries — one per numbered gap/conflict from the Phase 1 list:** + +- **Gap-filling** — one query per numbered Phase 1 gap +- **Conflict resolution** — queries that specifically test contradicting claims with version-specific terms +- **Primary-source deep dives** — fetch the primary directly (raw release notes / docs pages) for claims needing Tier 1 confirmation +- **Recency verification** — if not done in Phase 1, fetch the upstream changelog/releases NOW + +### Phase 2 output (before proceeding to Phase 3) + +**STOP and analyze Phase 1+2 combined results.** Update the gap/conflict list. Identify Phase 3 sources (preferred-source authors OR the tool-ecosystem fallback if no author covers the domain). + +## Phase 3: Preferred Sources OR Tool-Ecosystem Fallback (3+ queries) + +Objective: cross-reference findings against trusted thought leaders OR upstream maintainers. + +**Path A — a preferred-source roster exists.** If the consuming project maintains a preferred-sources roster (a list of trusted authors/domains in its `CLAUDE.md`, rules, or docs), identify 3+ entries relevant to the topic and launch 3+ queries using those author names as search qualifiers. + +**Path B — no roster, or no listed author covers the domain (typical for tool-ecosystem topics).** MUST cite all three: + +1. **Official maintainer** — the vendor's own social / GitHub / blog +2. **Upstream repo changelog or releases** — `gh api repos///releases` OR a raw `CHANGELOG.md` fetch this turn +3. **One recognized industry authority** — a top-voted community post or named-author practitioner blog + +See the discipline file's "Tool-ecosystem Phase 3 fallback" for the playbook. + +## Phase 4 (conditional): Additional follow-up + +If Phases 1-3 still have gaps, conflicts, or LOW-confidence claims: + +- Launch targeted queries to reach HIGH confidence on every remaining claim +- No limit on additional phases — iterate until every claim has HIGH confidence per the discipline file's "Confidence calibration" +- Regularly self-critique your approach and plan + +## Research principles (apply throughout all phases) + +- **Authoritative sources first** — Tier 0 (direct tool output) > Tier 1 (official docs fetched this turn) > Tier 2 (recognized authors, vetted blogs) > Tier 3 (training-data recall — NOT acceptable; must promote before acting). Tier table: the discipline file +- **Source code as spec** — when the topic is "how does library/implementation X behave" and X's source is reachable (GitHub, vendored dependency, package cache), READ the source — it outranks every doc about it, even across languages. For port/reimplementation topics, RESEARCH.md carries a semantics map: matched code excerpts (source ↔ target), gotcha notes, edge-case table +- **Version-aware** — always include version numbers in searches +- **Avoid SEO content farms** — down-rank listicles, repackaged content, vendor marketing pages. See the discipline file's "Source-quality red flags" +- **Main-context vs. agent trade-off** — prefer direct research when results inform decisions (avoids summarization loss). Use agents for parallel breadth within a phase +- **No shortcuts for small tasks** — a "quick config change" still gets the full discipline +- **No parallel MCP calls to the same stdio server** — that transport is serial. Run sequentially within a server, parallelize across different servers/tools +- **Graceful degradation** — if a tool category is unavailable this session, substitute equivalent coverage and document the gap; don't lower the bar + +## Outcome gate (run before presenting — MANDATORY) + +Research is not done when the phases finish — it's done when it passes this gate. Check what the run ACHIEVED against what good research requires, **grounded in the run's own artifacts** (the evidence table, the Phase 1/2 written gap lists, the fetch log) — NOT in your recollection of "did I do a good job." The same model that satisficed the bars runs this check, so a self-congratulatory recap rubber-stamps shallow work. Only artifact-grounded binary criteria bite. + +Each criterion is binary — read it off an artifact, not from memory. **Any FAIL returns to the named phase; do not present until all pass:** + +| # | Binary criterion | FAIL → | +|---|---|---| +| 1 | Every claim row has ≥1 Tier 0/1 source whose URL/command was captured THIS turn | Phase 2 — fetch the primary directly | +| 2 | No claim row's sources are ALL Tier-2 secondary | Phase 2 — get a primary | +| 3 | Every Phase 2/3 query traces to a numbered gap/conflict in a written analysis block | re-run the phase chained to the list | +| 4 | Every claim has ≥2 INDEPENDENT corroborators (not 2 cites of one upstream pool) | Phase 2 — widen sources | +| 5 | The Phase 2 falsification query ran and is recorded | Phase 2 — run it | +| 6 | Recency gate satisfied for every tool/library/API claim: the LATEST upstream changelog/release was fetched THIS turn and cross-checked against the claim (a stable project whose latest release is older than the 30/14/90d window passes once that release is confirmed as current; a major version bump since the cited doc invalidates it, first-party docs included) | Phase 2 — fetch changelog | +| 7 | Every accepted claim is HIGH confidence | Phase 4 follow-up — iterate to HIGH | +| 8 | Project fit checked against the consuming project's own conventions and stated direction | revisit before presenting | + +**Authoritative + consensus, reconciled:** the primary source is the SPINE of each claim; independent corroborators are the CONFIRMATION. When a top-ranked blog consensus contradicts the primary, the primary wins and the conflict is flagged explicitly — consensus never overrides a fetched authoritative source. Subagent returns are Tier 3 (synthesis), not corroborators, until their cited primaries are fetched this turn. + +Zero tolerance for false positives. A claim that can't pass the gate is a **Gap**, not a finding — report it as such, never launder it into the answer. Report the gate result (pass, or which criterion failed + what you re-ran). No limit on iterations. + +## Output Format + +Present research findings as: + +1. **Summary** — 2-3 sentence answer to the research question +2. **Evidence table** — `Claim | Sources (Tier 0/1 entries cite the URL/command fetched THIS turn) | Tier | Tool diversity | Confidence` +3. **Conflicts** — disagreements between sources (flagged explicitly; primary wins over blog consensus) +4. **Gaps** — claims not at ≥1 primary + 2 independent corroborators, OR LOW confidence (flagged for follow-up) +5. **Recency status** — primary-source age per tool/library claim +6. **Project fit** — how findings align with the consuming project's conventions and stated direction +7. **Outcome gate result** — pass, or which criterion failed and what was re-run + +If invoked standalone, present findings directly. If invoked as part of a larger workflow, findings feed into the subsequent planning step. + +## Final step: persist artifact for handoff + +Write the research output to `${user_config.notes_dir}//RESEARCH.md` — derive `` from the research topic or current branch name (kebab-case, ≤40 chars). If the consuming project declares its own working-notes convention (in its `CLAUDE.md` or rules), that convention wins over the default location. This file is the authoritative summary of the stage — a fresh session must be able to resume planning reading only this artifact. + +The artifact opens with a Task restatement, follows the Output Format above, and closes with a Next-stage-handoff (settled facts vs. open decisions for the planning step). + +If research spans many topics and RESEARCH.md exceeds ~2000 words, split overflow into sibling `research-.md` files in the same directory and keep RESEARCH.md as the index. + +**Intra-task pivot — delete stale research, don't layer.** If the approach you researched is abandoned mid-task for a different direction *before shipping*, delete the now-stale RESEARCH.md section and re-run the research on the new direction rather than keeping both — a superseded section misleads the planning step into planning against a dead approach. + +## What this skill does NOT do + +- **Does not make decisions** — presents verified evidence; the planning step (or user) decides +- **Does not write code** — researches only; execution is a separate step +- **Does not skip phases for "simple" topics** — task size does NOT reduce depth. All phases always run +- **Does not present training-data knowledge as current fact** — Tier 3 recall must be promoted to Tier 0/1 before claim acceptance + +## See also + +- `${CLAUDE_PLUGIN_ROOT}/skills/research/context/discipline.md` — source tiers, recency gates, broad-topic recipe, falsification recipe, tool-ecosystem fallback, confidence calibration, source-quality red flags, observed failure patterns diff --git a/plugins/discovery/skills/research/context/discipline.md b/plugins/discovery/skills/research/context/discipline.md new file mode 100644 index 000000000..7cbbc7fd1 --- /dev/null +++ b/plugins/discovery/skills/research/context/discipline.md @@ -0,0 +1,159 @@ +# Research discipline — sources, recency, falsification, broad topics + +Recipes and rationale behind the bars stated in the research skill's SKILL.md body, plus failure patterns observed in real sessions. + +## Source tiers (canonical for this plugin) + +| Tier | Source | Counts as | +|---|---|---| +| Tier 0 | Direct tool output captured this turn (` --help`, file Read, `gh api`, MCP tool result) | Strongest. Primary | +| Tier 1 | Official documentation **fetched this turn** with URL captured (vendor docs, GitHub source, language spec, RFC, upstream changelog) | Primary | +| Tier 2 | Secondary synthesized (AI-synthesis answers, Stack Overflow, recognized author blog, vetted vendor blog) | Secondary — corroborator only | +| Tier 3 | Synthesis without grounding (training-data recall, vague "I remember reading," subagent return without primary citation) | NOT acceptable for claim acceptance — must promote to Tier 0/1 first | + +## Source-tier ratio (per claim) + +Mandate: every accepted claim has **≥1 Tier 0/1 source PLUS ≥2 independent corroborators** of any tier. + +**Anti-pattern:** three AI-synthesis citations of three different secondary blogs = 1 Tier 2 source, not 3. They're synthesizing from the same upstream pool. Count INDEPENDENT primary sources, not citation count. + +**Tool-diversity per topic — MUST track in the evidence table.** Two sources both from one synthesis tool / both from one search engine / both from one author's blog network = 1 corroborator, not 2. + +## Recency gate (for libraries, tools, CLIs, APIs) + +Mandate: when the topic touches a library/tool/CLI/API/framework that ships releases, **one Phase 1 or Phase 2 query MUST fetch the LATEST upstream changelog or release notes this turn** and confirm the claims are current as of it. Acceptable forms: `gh api repos///releases/latest`, WebFetch on a raw `CHANGELOG.md` URL, the vendor's "What's New" page. The windows below bound how stale a cited doc may be before this cross-check is required — a stable project whose latest release is older than the window still passes once that release is confirmed to be the current one. + +**Tightening tiers:** + +| Topic class | Recency gate | +|---|---| +| Very active project (weekly releases, breaking changes, security-sensitive) | 14 days | +| Standard library / tool / CLI / API | 30 days | +| Architecture pattern / conceptual guide | 90 days | +| Foundational doctrine (DDD, SOLID, Hexagonal) | No recency gate — concepts don't drift | + +**Major version bump invalidates prior docs.** When the upstream repo moved `x.y.z` → `(x+1).0.0` since the doc was last updated, treat ALL prior docs as suspect — including first-party docs, which routinely lag a major release. Re-verify every behavior claim against the new release notes regardless of doc age. + +## Falsification step (mandatory Phase 2 query) + +Mandate: **exactly one Phase 2 query MUST attempt to falsify the leading hypothesis** from Phase 1. + +**Falsification query patterns:** + +- For a claim "X is canonical": query `"X deprecated"` OR `"X replaced by"` OR `"X removed in version"` OR `"alternative to X"` +- For a claim "X supports Y": query `"X does NOT support Y"` OR `"X Y incompatible"` OR the upstream issue tracker for Y limitations +- For a claim "use X for Y": query `"why X is bad for Y"` OR `"X anti-pattern"` OR a recognized author's critique +- For a claim "the convention is X": fetch the upstream maintainer's own latest writing OR the project's own CHANGELOG to check whether the convention shifted + +**Why mandatory, not advisory:** without an explicit "try to break it" step, every Phase 2 query confirms Phase 1 by accident. Confirmation bias is the default behavior — falsification has to be enforced. Falsification cannot be retroactive: it must be a deliberate "try to break this" query, not a query that happens to surface contradicting evidence. + +## Broad-topic auto-detect + +Mandate: when the research topic matches ANY of the triggers below, **double all phase minimums**: + +| Trigger | Example topic | +|---|---| +| 2+ vendors / 2+ tools / 2+ products named | "tool A vs tool B vs tool C memory conventions" | +| 3+ proper-noun product names in topic | "ORM + event store + messaging library integration" | +| Comparison topic ("X vs Y", "X or Y", "best of X/Y/Z") | "identity server A vs identity server B" | +| Multi-platform topic ("works on Windows, macOS, Linux") | "polyglot version manager comparison" | +| Migration / rebrand / convention-change topic ("from X to Y", "replaces X", "deprecates X") | "config format X replaces format Y" | + +**Doubled minimums:** 6+ queries per phase (was 3+), 12+ queries total (was 9+), 5+ distinct tool types across the topic (was 3+ per phase), 4+ independent Tier 0/1 sources per claim (was 1+). + +**Why:** multi-vendor topics have N times more drift surface. Each vendor ships its own changelog cadence, docs site, and naming-convention shifts. Single-vendor minimums under-cover the cross-vendor edges where hallucinations concentrate. + +## Query scaling — floors are not targets + +The per-phase minimums (3+ standard, 6+ broad-topic) are FLOORS to start from, not targets to stop at. Models satisfice to stated numbers, so a flat "3 per phase" reliably produces exactly-3 shallow phases. The corrective: make the query count a FUNCTION of the open-question count. + +| Phase | Query count | +|---|---| +| Phase 1 | ≥3 — broad seed; the floor genuinely applies because you don't yet know the gaps | +| Phase 2 | one per numbered gap + one per numbered conflict + the mandatory falsification query (≥3, no cap) | +| Phase 3 | one per remaining gap after Phase 2, against preferred-source / tool-ecosystem authorities (≥3) | +| Phase 4 | one per still-open gap or LOW-confidence claim, until all reach HIGH | + +Depth scales to the topic's actual open-question surface, not to a higher flat floor. A simple topic with 3 gaps runs ~3 Phase 2 queries; a gnarly one with 9 gaps runs 9. + +## Tool-ecosystem Phase 3 fallback + +Mandate: when no preferred-source author covers the topic's domain (typical for tool-ecosystem topics — AI coding tools, MCP servers, CI-platform specifics), Phase 3 MUST cite all three: + +1. **Official maintainer** — the vendor's own social / GitHub / blog +2. **Upstream repo changelog or releases** — `gh api repos///releases` OR a raw `CHANGELOG.md` fetch this turn +3. **One recognized industry authority** — a top-voted community post or well-known practitioner blog with the author named + +Don't skip Phase 3 because "no preferred author exists." + +## Primary-source-first protocol + +The "top of Google" is a ranking artifact, not an authority signal — SEO content farms outrank authoritative sources. The defense: never let the SERP BE the source. Three steps per claim. + +1. **Name the canonical home before searching** — the official docs site / repo / spec / changelog that OWNS the answer. If the consuming project ships a per-ecosystem source mapping or preferred-sources roster (check its `CLAUDE.md` and rules), use it; otherwise identify the ecosystem's official docs site, package registry, and upstream changelog yourself. Probe for a published doc-index first per "Machine-readable doc-index discovery" below. +2. **Fetch it directly** — with whatever direct-fetch tool is connected this session. Discover what's available from your tool list, the injected MCP server-instruction blocks, and the project's MCP registry; don't hard-depend on a specific server. Fetching the canonical home directly bypasses ranking entirely. +3. **Synthesis + SERP discover and corroborate only** — find the canonical home when unknown, surface independent corroborators. A synthesized answer points you AT the source; it is never the terminal source for an accepted claim. + +**Authoritative is not a waiver for corroboration.** Even the canonical doc still needs ≥2 independent corroborators and a freshness check — first-party docs routinely lag major releases. When the topic post-dates a major version, cross-check the canonical doc against the upstream changelog/release and treat any lag as a conflict to resolve. + +**Escalate on block, never downgrade.** A direct-fetch 403/429 means wrong fetcher, not vanished source. Escalation order: (1) a headless-browser URL reader if connected; (2) a managed scraping tool if available; (3) a synthesis tool forced to the blocked domain (domain-filter option). Only after those fail, fall back to secondary sources — and document the gap. + +**Negative claims need the primary fetched this turn.** "X is undocumented / removed / unsupported" requires fetching the canonical doc this turn and confirming absence — absence in training data ≠ absence in current docs. + +## Machine-readable doc-index discovery + +When the topic centers on a specific library / framework / site, probe for a published index BEFORE crawling by hand. **Probe-if-present**: absence is normal; fall through to the next. Each is a `{base}`-relative path on the canonical home. + +| Probe | Path | Nature | Use for | +|---|---|---|---| +| `llms.txt` | `/llms.txt` (or `/.well-known/llms.txt`) | **Curated** markdown index — maintainer hand-pick, deliberately partial | Fast orientation + page prioritization; NOT completeness | +| `llms-full.txt` | `/llms-full.txt` | Full doc content inlined | One fetch for a deep read of the whole curated set | +| `sitemap.xml` | `/sitemap.xml` | **Exhaustive** — every URL (50k URL / 50 MB cap per file) | Completeness — enumerate ALL pages | +| `sitemap_index.xml` | root | Index of child sitemaps when the site exceeds the cap | Large sites — follow the index to each child | +| `robots.txt` `Sitemap:` directive | `/robots.txt` | Names the sitemap location when non-default | Locating a non-default sitemap | +| `.md` page variant | append `.md` to a page URL | **Platform-specific** — native on some doc platforms; absent elsewhere | Markdown of one page without an HTML parse — PROBE first | +| In-repo docs tree | `gh api repos///contents/docs` or `git ls-tree` | Tier-0 enumeration when docs live in a Git repo | Doc source is a repo, not a site | +| Changelog feed | RSS/Atom, or `CHANGELOG.md` / releases | Release/change stream | Feeds the Recency gate above | + +**Curated ≠ exhaustive — the completeness trap.** `llms.txt` is the maintainer's hand-pick; using it alone for "go through every page" silently drops whatever was omitted. For full coverage, **sitemap enumerates, llms.txt prioritizes**. + +## Source-quality red flags + +Down-rank or refuse to cite: + +- Domain names that aggregate / repackage other content (sites that summarize without primary research) +- "Top 10 best X" listicles with no author named +- Articles where every claim links to other listicles (no terminal primary source) +- Vendor-comparison pages on a vendor's own marketing site (marketing copy, not technical doc) +- "AI-generated content" markers / disclaimers — treat as Tier 3 unless verified against Tier 0/1 + +Prefer: the vendor's own `/docs` subdomain, GitHub source code, RFCs, language specs, vendor changelogs, recognized author personal blogs with author name + bio. + +## Graceful degradation (missing tools) + +If a required tool category is unavailable this session (no synthesis MCP server, no web access), don't lower the bar — substitute and document: + +- Lost synthesis tool → substitute WebSearch + WebFetch + `gh api` for equivalent coverage +- Lost web access → flag the topic as `verification: incomplete — offline session`; do not edit code based on Tier 3 recall +- Document the gap in RESEARCH.md's `Gaps` section: which tool was unavailable, what alternative was used, residual risk + +## Confidence calibration + +The evidence-table `Confidence` column must be set per claim: + +- **HIGH** — 3+ independent Tier 0/1 sources agree; recency gate passed; falsification query failed to find counter-evidence +- **MEDIUM** — 3+ sources agree but mix of Tier 0/1 + Tier 2; OR 2 Tier 0/1 + open falsification gap; OR primary source > 30d old without changelog cross-check +- **LOW** — fewer than 3 sources; OR sources conflict; OR Tier 2-only consensus; OR primary source > 90d old + +Only HIGH-confidence claims are accepted (the outcome gate enforces this). A MEDIUM or LOW claim is a **Gap** — return to Phase 4 follow-up and iterate until HIGH, or report it as a gap; never a basis for code edits. + +## Observed failure patterns + +- **Synthesis tools give wrong versions.** AI-synthesis tools routinely assert wrong version numbers and hallucinate canonical conventions (a config path that "is canonical" but isn't). Always verify version-specific features empirically (`gh api repos///releases/latest`, an actual import/call test) — never trust secondary sources for version claims. A single direct fetch of the canonical doc falsifies this class. +- **Agent consensus can be unanimously wrong.** Multiple subagents agreeing is one source, not N — they share training priors. Verify claims empirically before shipping, especially env-var / tool-behavior claims. +- **Two sources can both be wrong.** Two sources parroting the same incorrect information is common. Count INDEPENDENT primary sources, not citation count. +- **Phases must be sequential.** Phase 2 MUST analyze Phase 1 results before launching. Running all phases in parallel produces redundant queries that miss the gaps Phase 1 would have revealed. +- **No parallel MCP calls to the same stdio server.** stdio transport serializes. Run queries sequentially within a server; parallelize across different servers/tools. +- **Subagent return = Tier 3 by default.** Even when a subagent's prompt mandates citation, the return is synthesis. Cited primary sources inside the return promote to Tier 1 once fetched/confirmed; bare claims stay Tier 3. +- **Cached doc URLs from prior turns are Tier 3, not Tier 1.** A fetch result from months ago that's now in the model's assumption set has aged out. Re-fetch on every research pass for the topic. +- **Convention / naming decisions need primary sources read directly** — synthesis summaries are insufficient when deciding on folder names, config patterns, or naming conventions; read 3+ primary sources.