repo-fleet-hygiene:audit confidence model and evidence rules have drifted from the collector

[kyle-sexton]

Severity: IMPORTANT (assigned by an audit agent; uncalibrated — not re-graded by a human).

The confidence model and the evidence rules have drifted from what the collector actually does. The
docs are the contract a consumer tiers decisions on, so drift here is a correctness problem, not a
tidiness one.

1. The "full" tier table covers half the finding kinds

skills/audit/SKILL.md:81 points at reference/confidence-model.md as the "Full tier/disposition
table". Measured:

$ grep -o "emit_finding [A-Z]* [a-z-]*" scripts/audit-fleet.sh | awk '{print $3}' | sort -u | wc -l
24
$ grep -c "^|" reference/confidence-model.md
14      # header + separator + 12 data rows

24 distinct finding kinds are emitted; the table has 12 rows. A consumer meeting an untabulated
kind has no documented disposition for it.

Likely mechanical cause: the reference files are dated 2026-07-17 while the 0.6.0/0.7.0 collector they
certify is dated 2026-07-23 — the docs predate the code that grew the finding set.

2. ACKNOWLEDGED is on the wrong axis and absent from the table

SKILL.md:91 requires an ACKNOWLEDGED group in the report, and the collector emits it (:648,
:1191). But ACKNOWLEDGED is a disposition — "this identity is known-inaccessible" — smuggled onto
the confidence axis, where the other values are HIGH/MEDIUM/LOW/UNKNOWN.

$ grep -c "ACKNOWLEDGED" reference/confidence-model.md
0

It appears nowhere in the model it belongs to. So the one axis a consumer is told to trust has a
fifth value with no definition, and it means something categorically different from the other four.

3. Evidence rule 3 misdescribes the actual mechanism

SKILL.md evidence rule 3 says merged-branch evidence comes from
gh pr list --repo <this-repo> --state merged --head <this-branch> per branch. The collector does not
work that way: it issues one batched --state merged --limit 200 query per repository (:973) and only
falls back to the per-branch --head form (:1033) when the privacy gate permits — which, for the
population that most needs it, it does not.

A reader reasoning from rule 3 will conclude every branch got an individual authoritative query. None
of the batch-window or gate behaviour is derivable from the documented rule.

4. Merge-strategy dependence is never disclosed

Under squash merges the local-ancestry-only predicate is near-inert. Measured on the audited fleet:
ancestry of the default branch held for 27 of 506 branches, because a squash rewrites the commits.
Neither SKILL.md nor confidence-model.md states that the LOW ancestry tier — or the tiering
generally — depends on the fleet's merge strategy.

Consequences a consumer hits without warning: git rev-list --count <tip> --not --remotes reads
non-zero for 410 of 506 branches (they are merged, squashed, and their remote refs auto-deleted),
and git cherry is one-directional (ALL-UPSTREAM proves content landed; NONE-UPSTREAM proves
nothing, since a squash collapses N commits so no individual patch-id matches).

Partially overlaps #1770's caveat, filed separately because it remains undocumented in the shipped
skill files, which is the defect.

5. Two tiers separated by a config knob, not by evidence

missing-worktree is MEDIUM and prunable-worktree is HIGH (:846-855;
confidence-model.md:12-13), but the difference between them is whether git's
gc.worktreePruneExpire window has elapsed — a user-tunable config value, not a difference in
evidence strength. Undisclosed.

6. Two of three reference files are unreachable from the hub

$ ls reference/
confidence-model.md  official-sources.md  security-review.md
$ grep -o "reference/[a-z-]*\.md" SKILL.md | sort -u
reference/confidence-model.md

official-sources.md and security-review.md have no pointer from SKILL.md, so progressive
disclosure never reaches them.

7. One label, two quantities

The header prints Repositories discovered: N (:1142) and the summary prints repositories=N
(:1191). These count different things and share an unqualified label.

Suggested fix

Documentation-side, no logic change: add the ACKNOWLEDGED row (or move it to a disposition axis
where it belongs), extend the table to all 24 kinds or scope its claim honestly, rewrite evidence rule
3 to describe the batch+gate mechanism, state the merge-strategy dependence, disclose that
missing-vs-prunable turns on gc.worktreePruneExpire, link the two orphaned reference files, and
qualify the two repositories labels. Regenerating the tier table from the collector's emitted kinds
would keep item 1 from recurring.

Environment

• repo-fleet-hygiene 0.7.0, marketplace install melodic-software
• 11 canonical checkouts, 506 local branches; fleet squash-merges throughout
• Observed 2026-07-29/30