Add vuln-triage Claude skill

[mostlikelee]

Related issue: N/A — tooling/skill addition

Summary

Adds .claude/skills/vuln-triage/SKILL.md, a Claude skill that encodes Fleet's vulnerability false-positive / false-negative triage workflow.

The skill:

• Routes a <software, CVE> pair to the correct scanner using an embedded software.source × platform → scanner table (NVD, OSV, OVAL, goval-dictionary, MSRC, macoffice, winoffice, CustomCVE).
• Runs tools/nvd/nvdvuln (Mode 1 or Mode 2) when the software is NVD-handled and parses its CPE/CVE output.
• Reconciles the CVE against multiple upstream sources (NVD, MITRE, GHSA, OSV.dev, vendor advisories) before proposing a fix, so upstream-data bugs are routed to feed overrides instead of Fleet logic changes.
• Surfaces a systemic-vs-one-off check before any override (CPE matching rule, CustomCVE entry, sanitize regex, feed override). When the systemic option changes scanner routing, the skill is required to update its own routing table alongside the code change.
• Verifies /tmp/vulndbs setup and points the engineer at fleetctl vulnerability-data-stream for missing feeds.

Why

Vuln-bug triage is a recurring task that spans 7 data sources and many override layers. The cognitive load is mostly route-finding. This skill captures that route-finding once so engineers don't reconstruct it each time, while keeping the engineer in the loop for any data-source or code edit (diagnose + propose, no auto-edit).

Notes for reviewers

• No production code changes. New file only at .claude/skills/vuln-triage/SKILL.md.
• File-path / line-number references in the skill are starting points — the skill instructs the agent to read the linked code on each invocation rather than trust the line numbers.
• The PR-template checklist below is mostly N/A for a skill-only change.

Checklist for submitter

• N/A — no user-visible product changes (Claude skill addition under .claude/skills/).
• N/A — no SQL, no code paths affected.
• N/A — no automated tests; verification is manual via the scenarios listed in the plan file.
• QA'd all new/changed functionality manually

Testing

Manual triage scenarios to validate after merge:

• Trigger sanity: a fresh chat saying "false positive on Foo flagged with CVE-X" suggests this skill; /vuln-triage invokes it explicitly.
• Dry-run on a recent vuln-related PR (e.g. recent commit touching server/vulnerabilities/nvd/cpe_matching_rules.go) lands at the same file/fix shape.
• Non-NVD scenario (Ubuntu python3-foo deb) routes to OSV, not nvdvuln.
• Cross-source disagreement scenario (NVD vs GHSA) is identified in the report.
• Systemic-vs-one-off scenario (npm CVE absent from NVD but present in GHSA) recommends the systemic option ahead of a CustomCVE entry.
• Missing /tmp/vulndbs produces a clear advisory message instead of a silent failure.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

_{Tip: disable this comment in your organization's Code Review settings.}

[Copilot]

Pull request overview

Adds a new Claude skill that documents Fleet's vulnerability triage workflow for false positives and false negatives across multiple data sources (NVD, OSV, OVAL, MSRC, Office, etc.). The skill provides a routing table mapping software.source × platform to scanners and prescribes a cross-source verification process before proposing fixes.

Changes:

• Adds .claude/skills/vuln-triage/SKILL.md with an 11-step triage workflow.
• Encodes a routing table for scanner selection and references key code paths.
• Establishes guardrails (cross-source verification, systemic-vs-one-off) and a diagnose-then-propose posture.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[dantecatalfamo]

I don't have a lot of experience with claude skills so I asked it to review the file. I don't know how accurate these are so take them with a grain of salt.

---

I verified the file/function references — all exist — and compared the skill against existing Fleet skills (fix-ci, review-pr, find-related-tests).

Factual error (must fix)

The OSV claim is wrong. The skill says:

OSV feeds are NOT included in vulnerability-data-stream.

But cmd/fleetctl/fleetctl/vulnerability_data_stream.go:106-111 calls osv.RefreshAll(ctx, dir) — OSV artifacts are part of the data stream. The warning in step 3 will send engineers chasing a non-issue. Drop
that sentence; mention that nvdvuln -sync also pulls OSV via the same osv.RefreshAll for parity.

Effectiveness issues

1. Edit in allowed-tools contradicts "Do not auto-edit." Step 10 says "Do not auto-edit. The user reviews each diff first," but Edit is granted. Either drop Edit (matches review-pr's tool list —
   Read/Grep/Glob only) or own the contradiction. As written, the model could honor allowed-tools and bypass the prose.

2. $ARGUMENTS is underused. The skill takes $ARGUMENTS then immediately walks the user through six required fields. Have step 1 parse whatever's in $ARGUMENTS first and only ask for what's missing —
   otherwise /vuln-triage Firefox 119 macOS CVE-2024-X false positive makes the agent ignore the input.

3. WebFetch on api.osv.dev will mangle JSON. WebFetch is HTML→markdown; raw JSON gets summarized weirdly. Replace that line with curl -s to ./tmp/<cve>.json then Read — and add the curl invocation to
   allowed-tools.

4. All 5 cross-source fetches are mandatory and heavy. Make them conditional on routing: only fetch GHSA when software.source is a language ecosystem (npm_packages, python_packages, etc.); only fetch
   MSRC when scanner is MSRC; only fetch vendor advisory when vendor matches Apple/Microsoft/Mozilla. Keep NVD + MITRE + OSV.dev as the always-on baseline.

5. allowed-tools bash patterns are too narrow for what's described. Bash(go run -tags fts5 ./tools/nvd/nvdvuln*) likely won't permit the full multi-flag command in step 4 (compare with fix-ci's broader
   Bash(go test *)). Test that the exact step-4 command actually passes the allow-list; if not, use Bash(go run -tags fts5 ./tools/nvd/nvdvuln:*) style.

6. "Diagnose + propose edit posture" is referenced without being defined. Step 10 opens with Per the 'Diagnose + propose edit' posture: as if it's a pre-existing concept. Inline the rule (one sentence) or
   drop the framing.

7. End-of-report references "fix-ci's end-of-run summary format." Cross-skill coupling — if fix-ci changes shape, this rots. Inline the format block directly (the field list right after is already most of
   the format anyway, so just remove the reference).

8. Step 9 git/gh searches are unbounded. git log --grep="<vendor>" -- server/vulnerabilities/ can dump years of history. Add --oneline -20 (or --since=2.years). Same for gh issue list — needs --limit 20 --state all.

9. "False positive" in step 1 isn't sharp enough. A real FP splits into:

   • (a) wrong CPE generated
   • (b) right CPE but NVD lists wrong CVE range
   • (c) right CPE+CVE but version comparison wrong

   Asking which one — or showing the disambiguation early — saves a triage round.

10. Routing-table self-maintenance is prose-only. The clever "update the routing table when scanner routing changes" rule has no enforcement. As a low-cost mitigation, add a <!-- Last verified at commit <SHA> --> line under the table so a stale table is visible at a glance during review.

Smaller stuff

• CLAUDE.md prefers ./tmp for generated files, but /tmp/vulndbs is an established Fleet path used by nvdvuln itself — fine to keep, no change needed.
• The description's trigger list is good but missing the literal /vuln-triage invocation that the PR test checklist relies on. Either add it or drop that test bullet.
• effort: high + model: opus is right for diagnosis work, but with cross-source fetches gated to be conditional (Add pretty-printing option for query results #4) you might be able to drop to effort: medium.

Verdict

The skill is technically accurate on file/function references and the routing table is a genuinely useful artifact. Top three fixes that would most improve effectiveness:

1. Correct the OSV claim
2. Resolve the Edit / no-auto-edit contradiction
3. Make $ARGUMENTS parsing work so the skill doesn't restart from scratch when invoked with structured input

[mostlikelee]

Thanks for the thorough review. Pushed cd26cb45f9 addressing most of it:

• Factual OSV fix — corrected; vulnerability-data-stream does sync OSV via osv.RefreshAll at cmd/fleetctl/fleetctl/vulnerability_data_stream.go:106. Also corrected the same claim in the PR description.
• $ARGUMENTS parsing — step 1 now parses $ARGUMENTS first and only asks for missing fields; echoes what it extracted for correction.
• WebFetch on JSON — OSV.dev now uses curl -s ... > ./tmp/<cve>.osv.json then Read; added the curl pattern to allowed-tools.
• Cross-source fetches — split into baseline (NVD/MITRE/OSV.dev — always) and conditional (GHSA only for language ecosystems, MSRC only when scanner is MSRC, vendor PSIRT only for first-party publishers).
• FP disambiguation — step 1 now asks (a) wrong CPE / (b) wrong CVE range / (c) wrong version comparison to steer steps 5–6.
• Bounded searches — git log --oneline -20 --since=2.years, gh issue list --limit 20 --state all.
• Routing-table staleness marker — added <!-- Routing table last verified at commit ... --> under the table.
• Diagnose + propose edit posture — inlined the definition; dropped the dangling reference. Also dropped the fix-ci end-of-run summary cross-reference (the field list right after was already the format).
• /vuln-triage literal — added to the description trigger list.
• Edit vs no-auto-edit — resolved earlier in cf3b27ed2a: rephrased step 10 so the gate is the explicit propose-first/approve step, not the tool-permission prompt.

Deliberately skipped:

• nvdvuln allow-list pattern — kept Bash(go run -tags fts5 ./tools/nvd/nvdvuln*). The trailing * matches the rest of the multi-flag command in step 4; will switch to :* if that turns out not to be true in practice.
• /tmp/vulndbs path — kept (your note agreed).
• effort: medium — kept high; even with gated fetches the reconciliation work is non-trivial.

[dantecatalfamo]

LGTM