[ci-scan-feedback] KPI Tracker

[github-actions]

Tracking quality of [ci-scan] (detection) and [ci-fix] (mitigation) issues, PRs, and loop-in comments since 2026-05-21. Updated every tick of ci-failure-scan-feedback.lock.yml. To raise a concern, comment here or on any [ci-scan]/[ci-fix] issue/PR; the next tick reads in-scope feedback and either opens a [ci-scan-feedback] PR with prompt edits or pushes to the existing one.

Snapshot — 2026-06-28T10:24Z

Activity (last 7d)

stream	opened	good	wrong/unmerged
Scanner KBE issues	6	3	0
Fixer [ci-fix] PRs (confident)	0	0	0
Fixer [ci-fix] PRs (help-wanted)	0	1	1
Fixer loop-in comments	1	0 engaged	—

"good" = KBEs closed completed, PRs merged, loop-ins with a maintainer reply. "wrong/unmerged" = KBEs closed not_planned/duplicate, PRs closed without merge (note: a help-wanted PR closed unmerged is expected, not necessarily a miss).

Scanner quality (last 30d)

metric	count	rate
KBE closures	46	—
Wrong KBE closures	11	23.9%
Maintainer rejection comments	0	—
Duplicate KBEs	7	—

Fixer quality (last 30d)

metric	count	rate
Fix PR closures	5	—
Fix PRs merged	1	—
Fix PRs rejected (maintainer pushback)	1	20%
Loop-in comments (posted / engaged)	2 / 0	—
Loop-in / help-wanted mis-attributions	0	—

Merged PRs: #129652 (help-wanted, TensorPrimitives trig tolerance) merged by tannergooding on 2026-06-22.

Rejection detail: #129360 (help-wanted, PGO codegen workaround) — MihaZupan (MEMBER): "I don't see a point in trying to workaround a codegen bug like this." Addressed by open feedback PR #129611 which adds a pipeline-category gate.

Neutral closures: #129235 closed by jkotas as superseded by merged #129652 ("not an exact duplicate, just different attempt to fix the same issue"). #129328 and #129314 closed by kotlarmilos — triaged-away (Mono/low-priority), not a rejection.

Outage signals (analyzed CI)

signal	threshold	24h	7d	status
New-KBE burst	day > 2x trailing 30d median (min 3)	4 / median 1	peak day 4	🔴
Build-break spike	>= 2 in any 24h	1	max 1 per day	🟢
Multi-pipeline outage (distinct legs proxy)	>= 3 distinct legs in 24h	4 distinct	4 distinct (peak day)	🔴
KBE re-filed after maintainer close	any in 7d	0	0	🟢
Scanner wrong-closure rate (30d)	>= 30% with scan_closed_30d >= 10	—	23.9%	🟢
Fixer rejection rate (30d)	>= 30% with fix_closed_30d >= 5	—	20%	🟢

details: New-KBE burst — 4 KBEs filed on 2026-06-28: #129936 (NativeAOT linker alignment arm64), #129937 (JIT assert "No IL header" jitstress2), #129938 (AsyncProfilerTests buffer flushes jitstress), #129939 (KmacXof128 invalid key length PGO). All are distinct failure shapes across different pipelines.
details: Multi-pipeline outage — linux-x64 Release (libraries-pgo), windows-x64 Release (libraries-jitstress), linux-arm64 Checked NativeAOT_Pri1 (nativeaot-outerloop), coreclr Pri1 windows x64 checked jitstress2 (jitstress) all red on 2026-06-28. These are independent failures across 4 pipelines; the burst is CI instability, not a scanner over-filing issue.

---

Workflow health

Workflow	Last 2 runs	Status
ci-failure-scan	28307610884 (✅ success, 06-28), 28289889790 (❌ failure, 06-27)	✅ Latest run healthy
ci-failure-fix	28307740562 (✅ success, 06-28), 28290456197 (✅ success, 06-27)	✅ Healthy

---

Open feedback PR

#129611 — [ci-scan-feedback] Add pipeline-category gate to fixer prompt (draft, 4 commits). Addresses the #129360 rejection by adding a gate that blocks ci-failure-fix from attempting workaround fixes on JIT/GC/PGO stress pipeline KBEs. Pending review by kotlarmilos.

---

Trend (last 3 ticks)

Tick	scan_closed	scan_wrong_rate	fix_closed	fix_reject_rate	KBE burst	Re-file
2026-06-28	46	23.9%	5	20%	🔴	🟢
2026-06-25	55	25.5%	4	n/a	🟢	🔴
2026-06-21	52	25.0%	3	n/a	🟢	🔴

Trend notes: Scanner wrong-closure rate continues to improve (25.5% → 23.9%). The X509 re-file signal has cleared (no new re-files in the current 7d window). Fixer rejection rate is now computable (5 closures reached threshold) at 20% — well below the 30% alarm threshold. New KBE burst and multi-pipeline outage are CI health signals (4 distinct new failures across 4 pipelines in one day), not scanner quality issues.

Note

🔒 Integrity filter blocked 4 items across KBE dedup searches (issues #129832, #129876, #114222, #117138). These could not be read and were skipped.

Note

🔒 Integrity filter blocked 1 item

The following item was blocked because it doesn't meet the GitHub integrity level.

• #128501 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by CI Outer-Loop Failure Scanner — Feedback · ● 9.9M · ◷

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.