Repository navigation
feat(bug-report): add scan — proactive two-stage bug-finding skill - #3208
Conversation
Contract for the new /bug-report:scan proactive bug-finding skill: interview + two adversarial audit rounds resolved 18 decisions (register clean, brief cross-check ok). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
Cursor ladder for the no-filing mode, label-taxonomy dual-axis wording, report frontmatter fence, trigger disambiguation, and same-PR setup and config-cascade obligations. Register: 25/25 answered, gates clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
Five-phase plan (scan skill, setup check|apply + cascade row, metadata, local gates, PR/prune/merge) with the four deferred execution-shape decisions resolved and tagged. Fresh-context stress-tests in flight; fixes will follow as amendments. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
Folds 21 verified findings from the fresh-context plan review and devils-advocate pass: Phase 0 base refresh (branch was 161 commits stale; bug-report 0.7.4 on main), corrected gate invocations, --track flag rename, operator/cadence/summary frontmatter, shared reference/config.md contract home, cursor provenance marker and degrade paths, and the squash-only PR gate requirements. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
`/bug-report:scan` hunts unobserved bugs in resting code — targeted at a path/feature/diff, or rotating a lane on a bare invocation — and verifies every candidate through a separate fresh-context default-refute gate before reporting. Read-only toward the target repo; filing sits behind `--track`. - SKILL.md: verb contract, cursor ladder (tracker provenance -> persisted report cursor -> date-derived floor), per-run budget, two-stage pipeline, dogfood-filing beats, handoffs. - context/lenses.md: five hunter lens contracts in the four-part subagent shape, evidence-quote and no-candidate rules, bundled generic default lanes. - context/verification-gate.md: default-refute stance, falsification routes, reproduced vs verified-by-reading labels, retained-refuted output contract. - context/findings-report.md: report shape, refuted tail, cursor metadata block; never declares `type: review-findings`. - reference/config.md: the single home for the `.claude/bug-report.md` key contract (lanes, filing_posture, output_dir partition rule). - evals/evals.json: six cases covering targeted, rotation, --track, read-only, routing, and --dry-run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
…nfig The plugin now owns a writable artifact — the tracked `.claude/bug-report.md` lane config `/bug-report:scan` reads — so the check-only carve-out no longer describes it. Setup takes the narrow-write shape instead: `apply` bounded to that one file, while `output_dir` stays a native `userConfig` surface handled the check-only way through Claude Code's own configuration prompt. - setup/SKILL.md: description and body rewritten off the userConfig-only carve-out; `check` keeps the existing output_dir probes intact and gains a per-layer report of the cascade file (presence, provenance, version-control verdict, inert unknown keys); `apply` drafts lanes, writes only the team layer conservatively, verifies by re-reading, and recommends — never writes — the overlay gitignore line. Keys, layers, and merge semantics are cited from `reference/config.md` rather than restated. - setup/evals: three apply/check cases added alongside the existing two. - config-cascade: `bug-report` Implementers row — all three layers, conforms, with merge semantics and the output_dir partition declared beside it. - plugin-data-report-keying: `bug-report:scan` recorded as a slug-keyed writer of reports plus the cursor metadata it reads back. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
The plugin gained a skill (`scan`) and a rewritten `setup` in this branch but still declared 0.7.4 with no release note, no README coverage, and a stale generated cheat sheet. - `plugin.json`: 0.7.4 -> 0.8.0; keywords gain `scan` and `bug-hunting` (same shape the fleet uses for a skill-adding minor). - `CHANGELOG.md`: a `## [0.8.0]` entry above the preserved 0.7.4/0.7.3/0.7.2 headings, covering the scan skill's two-stage pipeline, its read-only verb contract and `--track` filing, the stateless cursor ladder and per-run budget, `reference/config.md` as the key contract's single home, setup's move from check-only to `check | apply`, and the two convention-registry rows. - `README.md`: a three-row skills table, a scan section with its flag table and the two properties a reader needs before relying on it, and a Configuration section rewritten around two surfaces — the personal `output_dir` userConfig and the tracked `.claude/bug-report.md` — replacing the now-false userConfig-only and check-only claims. - `docs/SKILL-CHEAT-SHEET.md`: regenerated (scan's operator-cadence row). The catalog regenerated to no change; plugin descriptions are unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
… own `## Usage` read as the plugin's usage while it documents `/bug-report:write`'s flags alone; the scan section beside it made that ambiguous. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
Close-out per the contract-slice convention: the approved contract is published in PR #3208's description, which is the durable pointer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
|
Claude finished @kyle-sexton's task in 1m 47s —— View job Security review
Note on the requested procedure: the custom instructions asked me to invoke Scope of this PRThis PR is documentation/skill-definition only: it adds FindingsNo security vulnerabilities identified. Specifically checked:
One point worth naming (design note, not a vulnerability in this PR): like any agent skill that reads arbitrary repository content for a fresh-context subagent to reason over, hunter/verification prompts inherently process untrusted repo text (comments, strings, docstrings). The design already mitigates the main risk by keeping hunters strictly read-only and separating the recall stage from a default-refute verification stage — there's no path from "content in a file" to a write/file action without the explicit |
|
Last security-reviewed head: |
|
Claude finished @kyle-sexton's task in 1m 45s —— View job Claude finished the security review
Skip gate check: PR is open, so I reviewed the diff for security-relevant surface. Files reviewed ( Result: no security issues foundThis PR adds Markdown skill/prompt definitions and documentation — no application code, no
No CRITICAL/IMPORTANT/SUGGESTION findings to report. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 52f6d3e5a8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…up tracking probe — review findings
Four verified reviewer findings against the scan and setup skill contracts:
1. (P1) scan Step 5 gated filing only on work-items presence plus a tracker
binding, so `--track` would file even under the default `manual-only`
filing_posture the config reference declares. Step 5 now opens with a
posture gate resolved from the cascade (bundled default `manual-only`),
printing one "filing skipped" notice and stopping at the report — the same
degrade shape as an absent work-items. The `--track` mode row notes the
posture dependency.
2. (P2) scan cursor ladder rung 2 read the newest report blindly, but
/bug-report:write's reports share that directory and carry no cursor block,
and a targeted scan must never advance rotation. Rung 2 now searches
backward for the newest report carrying a valid rotation cursor block and
falls through to rung 3 when none does.
3. (2b) rung 2 also hardcoded ${CLAUDE_PLUGIN_DATA}/bug-reports/<project-slug>/,
stranding the cursor whenever output_dir is configured. It now resolves the
directory by the same precedence persistence uses (write's Step 4).
context/findings-report.md is made consistent: the cursor block belongs to
rotation-mode reports (bare or --lane); a targeted run omits it and prints a
one-line non-advancing marker instead.
4. (P2) setup `check` step 3 used `git check-ignore` alone — an ignore probe
that returns the same empty output for a present-but-untracked file as for a
healthy tracked one, so check could bless a config teammates never receive.
It now pairs check-ignore with `git ls-files --error-unmatch` (non-zero exit
on a present file = FAIL). `apply`'s post-write verification mirrors the
pairing but does not FAIL a freshly scaffolded file: it states the
stage/commit obligation instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
|
Claude finished @kyle-sexton's task in 1m 12s —— View job Claude finished the security review
Skill invocation note: Result: no security issues foundThis PR remains Markdown skill/prompt/doc content only — no application code, no
On the Codex review comments: all three No CRITICAL/IMPORTANT/SUGGESTION security findings to report. |
|
Codex review dispositions — all four findings verified against the shipped text, confirmed real, and fixed in e63c9cb:
Both skills re-pass Generated by Claude Code |
|
Claude finished @kyle-sexton's task in 2m 19s —— View job Claude finished the security review
Skip gate check: PR is open. The head SHA ( Diff reviewed: Result: no security issues foundThis remains a documentation/skill-definition-only change — no application code, no CI/workflow changes, no dependency or permission changes. I specifically re-checked:
All four prior Codex findings (P1 filing-posture bypass, P2 No CRITICAL/IMPORTANT/SUGGESTION security findings to report. |
…w fixes Seven leftover sentences still described the pre-fix behavior: - findings-report.md: the all-refuted and zero-findings forms now name the cursor block as the rotation-run form, with the targeted run's no-cursor line called out beside each. - scan/SKILL.md: Step 4's report shape, the pipeline's filing aside, and the frontmatter description now carry the mode and filing_posture qualifiers. - scan/evals: the --track case declares filing_posture: allowed in its prompt so its filing expectations are reachable against the bundled manual-only default, and gained a posture-gate expectation; rung 2's wording now matches the backward search for a valid rotation cursor block. - CHANGELOG 0.8.0: rung 2 described as the newest report carrying a valid rotation cursor block, resolved through persistence's directory precedence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
|
Claude finished @kyle-sexton's task in 56s —— View job Claude finished the security review
Skill invocation note: Diff reviewed: Full PR diff ( Result: no security issues foundThe new commit (
This PR remains documentation/skill-definition content only (Markdown prompt contracts + JSON eval fixtures) — no application code, no CI/workflow changes, no dependency changes. All four prior Codex findings (P1 filing-posture bypass, two P2 cursor-selection issues, P2 weak tracking probe) were security-adjacent design/correctness issues, not vulnerabilities, and remain fixed as of this head per the diff. No CRITICAL/IMPORTANT/SUGGESTION security findings to report. |
|
Claude finished @kyle-sexton's task in 41s —— View job Code review — skipped (PR merged)
No further action taken. |
|
Claude has reviewed this PR 1 time. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
No linked issue ## Summary Renames the `bug-report` plugin to `bugs` — with `scan` (landed in #3208) beside `write`, the plugin's identity is the front half of the bug lifecycle (*find them, report them*), and the old name described only the second half. Follow-up to #3208 by owner request: hard rename, no aliasing/retirement machinery, breaking changes accepted. Plugin 0.8.0 → 0.9.0. ## Fix - **`plugins/bug-report/` → `plugins/bugs/`** (15 tracked renames); skills are now `/bugs:scan`, `/bugs:write`, `/bugs:setup`; marketplace entry, `.claude/settings.json` enablement, and the skill-leaf-name registry updated. - **Tracked team config surface renames with the plugin**: `.claude/bug-report.md` → `.claude/bugs.md` (same keys, cascade, and merge semantics per `reference/config.md`); convention registries (config-cascade, plugin-data-report-keying) updated. - **Cross-fleet live references swept**: `claude-ops:known-issues` (0.37.1) and `work-items:track` (0.39.19) with their own changelog entries per the published-version-reuse gate. - **Deliberately unchanged**: the persisted report frontmatter `type: bug-report` (storage-format identifiers stay stable across renames — ADR 0013), the `bug-reports/` plugin-data subdir literal, the `'bug-report this'` trigger phrase, and historical changelog/upstream/topic-slice records, which retain the old name as immutable history. The old-name marketplace tag is kept as a search alias. ## Verification Fresh-context verifier audited the rename against 8 criteria: file-census parity (15/15 carried), manifest/marketplace/settings state, byte-identical changelog history under the new 0.9.0 entry, a classified residual-reference sweep (only deliberate keeps remain), cross-fleet surface updates with the keying-convention citation still resolving, and gates. Its one FAIL (unversioned cross-plugin edits) and one scope flag (`type:` identifier drift) are fixed in d67519b. Local gates all exit 0: `validate-plugins.sh`, all four `check-changelog-parity.sh` modes, `check-changed-skills.sh origin/main` (5 skills PASS), catalog + cheat-sheet `--check`, markdownlint, typos, editorconfig-checker. ## Related - Refs #3208 (the scan skill this rename reframes the plugin around) - ADR 0013 (storage-format identifiers stable across renames) — followed here - Conventions: `docs/conventions/config-cascade/`, `docs/conventions/plugin-data-report-keying/` 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX --- _Generated by [Claude Code](https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
2 participants
No linked issue
Summary
Adds
/bug-report:scan— the marketplace's first proactive bug-finder. Every existing correctness-finding producer is gated on a diff, an observed failure, a factual claim, a test file, or a comment marker;scanhunts unobserved defects in resting code (targeted"find a bug in <X>", or bare lane rotation for a daily routine), verifies candidates adversarially before reporting, and hands verified findings to the plugin's existing report/filing machinery. Pluginbug-report0.7.4 → 0.8.0.Fix
plugins/bug-report/skills/scan/— two-stage pipeline: recall-biased per-lens hunter subagents (5 lenses: same-unit contract-vs-body mismatch, boundary/edge-case, cross-file consistency drift, state/concurrency hazards, git-hotspot-guided) then a separate fresh-context default-refute verification gate; findings labeledreproducedvsverified-by-reading; refuted candidates retained in the report, never silently dropped. Read-only on bare invocation; per-run budget (3 verified findings / 10 candidates per wave / refill cap 2); stateless three-rung lane cursor (tracker provenance search → persisted-report cursor metadata → date-derived index).--trackfiles verified findings as raw work-items intake (dogfood-filing beats, dual-axisneeds-triagefrom the live label set, no label creation) behind an explicit flag only;--dry-runpersists nothing.plugins/bug-report/reference/config.md— single home for the tracked.claude/bug-report.mdkey contract (concatenatinglaneswith empty-list opt-out; nearest-winsfiling_posture;output_dirstays native userConfig).setupextended check-only →check | apply— narrow-write shape,applybounded to the tracked.claude/bug-report.md; existingoutput_dircheck preserved byte-identical.docs/conventions/config-cascade/README.mdanddocs/conventions/plugin-data-report-keying/README.md.scan/bug-huntingkeywords), CHANGELOG 0.8.0 entry above ten preserved headings, README (3-skill table, scan section, two-surface Configuration), regenerateddocs/SKILL-CHEAT-SHEET.md(catalog needed no delta — plugin description unchanged).Verification
Every phase was implemented by a scope-fenced worker and independently verified by a fresh-context phase-verifier against binary acceptance criteria (Phase 1: 10/10 PASS, Phase 2: 10/10 PASS, Phase 3: 8/8 PASS). Local gate pass, all exit 0:
check-changed-skills.sh origin/main(both skillsCHECK-SKILL PASS — 0 errors),check-changelog-parity.shall four modes,validate-plugins.sh(catalog + cheat-sheet in sync, 72 manifests),check-jsonschemaon both evals files, markdownlint (0 issues), typos, editorconfig-checker, actionlint, gitleaks. Design was locked via a 25-question interview ledger, two adversarial audit rounds (4 fresh validators), verifier-PASSed exploration/research artifacts, and a dual fresh-context plan stress-test (21 findings folded in).Related
docs/PLUGIN-PHILOSOPHY.md(verb table, setup narrow-write),docs/conventions/config-cascade/,docs/conventions/plugin-data-report-keying/,plugins/work-items/reference/dogfood-filing.mddocs/topics/bug-finding-skill/slice is pruned pre-merge per the contract-slice convention, with this PR body as the durable pointer.Approved PLAN.md (planning contract — Brief + Plan)
The complete contract lives in
docs/topics/bug-finding-skill/PLAN.mdon this branch's pre-prune history (commit 52f6d3e). Key content:Brief — TLDR: new
/bug-report:scanskill (proactive, general-purpose bug finding — on demand, targeted, or a daily routine's single bounded pass); read-only find → verify → report with two-stage precision; targeted scoping + lane rotation with per-run budget; filing behind an explicit argument only, always raw intake; setup writes a tracked config-cascade project file; evals + skill-quality + CI gates.Constraints (abridged): verb-table read-only contract on bare invocation; no sibling-plugin imports (presence-gated composition); discovering agent never grades its own findings; dogfood-filing conformance with live-label-set
needs-triageresolution; durable cursor never in.work/(three-rung stateless ladder); findings report never declarestype: review-findings; refuted candidates retained; refill rounds capped; cross-plugin composition by skill invocation or artifact contract only; eight skip-when fences (review:*, review:security-review, debugging:debug, codebase-health:audit, code-tidying:tidy, work-items:scan-todos, testing:audit, mutation-testing:audit).Out of scope (explicit deferrals): in-run fixing; cross-repo scanning; formal loop-lane convention adoption; born-briefed self-filing (pending dogfood-filing owner-contract amendment); detector-findings-format persistence; a standalone
bug-findingplugin.Execution: Phase 0 base refresh (branch was 161 commits stale) → Phase 1 scan skill (commit 2132c63) → Phase 2 setup + registry rows (75015bd) → Phase 3 metadata + regen (a0f6863, d4c261e) → Phase 4 local gate pass → Phase 5 PR/prune/squash-merge. Each phase carried mechanically verifiable sanity checks and an independent fresh-context verifier PASS before its
[DONE]mark.Key tagged decisions: verification gate as inline-prompted fresh-context subagents (codebase-health precedent); filing flag
--track(write owns--file);workflow-stage: operator+cadence: daily+summary(cheat-sheet generator contract); scan-filed items carry the body provenance lineFiled by /bug-report:scan (lane: <name>); same-day zero-state concurrency accepted for V1 (dedupe absorbs); variant-analysis seeding deferred post-V1.🤖 Generated with Claude Code
https://claude.ai/code/session_01JMRQbnTGgw3zxsd1fUDstX
Generated by Claude Code