The second sub-issue of #9726, which asks to "publish the series on /fairness with a methodology note". The series ships; the note does not.
State
- Computed and persisted from
decision_records — src/review/automation-rate.ts. Done.
- Published at
/v1/public/stats under automationRate, and rendered on /fairness with a weekly table. Done.
- Methodology note —
fairness-methodology.mdx does not mention automation at all. Missing.
Why the gap matters more here than for the sibling metrics
Every other figure on /fairness has its definition written down. This one is the metric the 97% target is stated against, and its definition contains three choices a reader cannot guess and would plausibly assume the other way:
- Non-deciding verdicts are excluded from both halves, not counted as automated.
action also carries label, update_branch, approve and the error classes; counting those would let work the gate never ruled on read as an automated decision.
- A PR that was held and later merged is manual. Scoring the final disposition instead would let the rate be inflated by holding everything and merging it by hand.
- Three human signals count, not just
hold — a named reevaluation_actor, and a reevaluation_reason of maintainer_request, also make a PR manual.
There is also a backfill horizon the page surfaces as a per-week basis field without explaining it anywhere. The re-evaluation provenance fields arrived in migration 0204 on 2026-07-29; weeks before that can only observe hold, so they can only under-count manual work and therefore only over-state automation. A reader comparing this quarter to last needs to know that, and right now the only place it is written down is a source comment.
Acceptance
- The methodology page documents the buckets, the three deliberate choices above, and the horizon — with the same numbers and language the code uses.
/fairness's automation section links to it, as its sibling sections already link to their definitions.
The second sub-issue of #9726, which asks to "publish the series on /fairness with a methodology note". The series ships; the note does not.
State
decision_records—src/review/automation-rate.ts. Done./v1/public/statsunderautomationRate, and rendered on/fairnesswith a weekly table. Done.fairness-methodology.mdxdoes not mention automation at all. Missing.Why the gap matters more here than for the sibling metrics
Every other figure on
/fairnesshas its definition written down. This one is the metric the 97% target is stated against, and its definition contains three choices a reader cannot guess and would plausibly assume the other way:actionalso carrieslabel,update_branch,approveand the error classes; counting those would let work the gate never ruled on read as an automated decision.hold— a namedreevaluation_actor, and areevaluation_reasonofmaintainer_request, also make a PR manual.There is also a backfill horizon the page surfaces as a per-week
basisfield without explaining it anywhere. The re-evaluation provenance fields arrived in migration0204on 2026-07-29; weeks before that can only observehold, so they can only under-count manual work and therefore only over-state automation. A reader comparing this quarter to last needs to know that, and right now the only place it is written down is a source comment.Acceptance
/fairness's automation section links to it, as its sibling sections already link to their definitions.