[docs] metrics-baseline denominator note misstates the ~12% figure's basis — claims consistency where populations differ #1419

Story

As a maintainer who will compare the post-rollout noise measurement against the pre-rollout baseline,
I want docs/metrics-baseline.md to state the ~12% figure's actual denominator rather than an incorrect claim that it matches the classifier's,
so that the first measured value is interpreted correctly instead of a real population difference being explained away as agreement.

Problem

docs/metrics-baseline.md (landed in #1414, story #1411) closes its denominator-scope note with:

"The AC #4 ~12% figure cited in issue #1411 was also measured over first-party markers only (it pre-dates the classifier and was an estimate, not a live all-comments count), so the denominators are consistent."

That claim is incorrect. The ~12% figure was derived over all PR comments, third-party reviewer bots included. Its provenance: a 10-PR sample (#1355, #1366, #1372, #1359, #1351, #1357, #1356, #1347, #1346, #1345) where total comments (issue + review) = 180 and est. no-action/advisory = 21 → 21/180 ≈ 12%. That 180 counted every comment on those PRs — CodeRabbit, Qodo, CodeAnt, SonarCloud, Codex and Gemini among them — and the no-action classification was a phrase scan ("no action", "no issues", "LGTM", "no blocking", "advisory") across all of them, not a marker-bearing subset. In the same sample, 254/254 comments+reviews were machine-authored and 0 were human, which is only meaningful because third-party bots were in scope.

The classifier's denominator is first-party marker-bearing comments only. So the two populations genuinely differ, and the baseline doc now asserts they do not.

Why this matters. The note's purpose was to satisfy #1411 AC #4 — "any discrepancy with those figures is explained rather than quietly overwritten." The mechanism is right and the rest of the note is good (it names the excluded bots, points at the reviewer scorecard, and asks that future all-comments comparisons be labelled). But the concluding claim pre-emptively explains away any divergence on the first scheduled run as "consistent denominators", when a divergence is in fact the expected consequence of measuring a narrower population. That converts a real measurement correction into an apparent non-event — the outcome AC #4 exists to prevent, reached by an incorrect factual claim rather than by silence.

This is a documentation-accuracy fix. No classifier change is implied — first-party-only remains a defensible primary metric (it is the part we control), and F1/F2 of the review on #1414 were correctly fixed before merge.

Acceptance Criteria

1. The denominator-scope note in docs/metrics-baseline.md states the ~12% figure's actual basis: all PR comments in the sampled window, third-party reviewer bots included, classified by phrase scan — and records the sample (10 PRs, 21/180).
2. The sentence asserting the denominators are consistent is removed or corrected; the note states plainly that the classifier measures a narrower population than the ~12% estimate, so the first measured value is expected to differ, and that such a difference is a population change and not a regression or an improvement.
3. The note gives the reader a rule for interpreting the first scheduled run: what would count as a genuine measurement correction versus the known population difference.
4. Either a second all-comments noise figure is reported alongside the first-party metric, or the doc states explicitly that no all-comments series is maintained and names the reviewer scorecard as the only third-party view — so a future reader cannot mistake the first-party series for total reviewer noise.
5. The correction is recorded as a dated append (the doc's own convention for measurement corrections), not a silent edit of the original text.

Tasks / Subtasks

• Correct the denominator-scope note per AC test issue from agent #1–Add @claude delegation, auto-merge, and rebase handling #3, keeping the parts that are already right (named excluded bots, scorecard pointer, labelling requirement). (AC: test issue from agent #1, Go-live improvements for PR review agent #2, Add @claude delegation, auto-merge, and rebase handling #3)
• Decide and document the all-comments question — either add the second figure or state that none is maintained. (AC: Optimize review: small-PR and incremental fast paths #4)
• Append the correction with a date rather than rewriting history in place. (AC: feat: add Copilot engine support via REVIEW_ENGINE toggle #5)

Dev Notes

• Provenance for AC test issue from agent #1, quotable directly: sample = the 10 PRs listed above; total comments (issue + review) = 180; est. no-action/advisory = 21; bot/agent-authored comments+reviews = 254, human = 0. The per-PR bot column spanned codeant-ai, coderabbitai, qodo-code-review, sonarqubecloud, chatgpt-codex-connector, gemini-code-assist.
• Live corroboration that third-party volume is material and would be excluded: on PR feat: implement issue #1403 — [Phase 1] Write docs/agentic-interaction-model.md — the event-first standard, three trigger classes, the GITHUB_TOKEN bridge rule, and the timer contract #1413 within one hour, the comment stream included a Codex quota notice, a Qodo "Bugs (0)" summary, a CodeAnt review-status table and a SonarCloud quality-gate banner — all classify non-agent under the current classifier, all are clutter a human scrolls past.
• Do not widen the classifier to fix this. First-party-only is the right primary metric; the defect is the doc's claim about the baseline, not the code's scope choice.
• The rest of docs/metrics-baseline.md is sound — the bimodal convergence split, the "record a dated row, never silently overwrite" convention, and the no-new-cron property should all be preserved.

Project Structure Notes

docs/metrics-baseline.md only. No script, workflow, or test changes expected.

References

• docs/metrics-baseline.md (denominator-scope note, landed in feat: implement issue #1411 — [Phase 4] Capture the pre-rollout baseline for the three success metrics, including a new no-action-comment noise metric #1414)
• [Phase 4] Capture the pre-rollout baseline for the three success metrics, including a new no-action-comment noise metric #1411 AC Optimize review: small-PR and incremental fast paths #4 (reproduce-or-correct the baseline figures, discrepancy explained not overwritten)
• scripts/lib/comment-noise.sh (cn_marker_pattern — the first-party-only denominator)
• scripts/reviewer_report.sh (the third-party reviewer scorecard, the separate view the note cites)
• Review thread on PR feat: implement issue #1411 — [Phase 4] Capture the pre-rollout baseline for the three success metrics, including a new no-action-comment noise metric #1414 (finding F3, the origin of the note)

Likely target surface

• docs/metrics-baseline.md

Filed after PR #1414 merged. F1 and F2 from that review were correctly fixed before merge; F3's mechanism was implemented but its concluding factual claim is wrong. Filed under epic #1402.