[prompt-analysis] Copilot PR Prompt Analysis - 2026-06-21

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis — 2026-06-21

Summary

Analysis Period: Last 30 days
Total PRs: 1,000 | Merged: 795 (79.5%) | Closed: 202 (20.2%) | Open: 3 (0.3%)

Overall success rate is stable at ~79–80%, consistent with the past 10 days of historical data.

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Bug Fix	663	519	141	79%
Testing / Linting	158	135	23	85% ✅
Feature Addition	102	80	22	78%
Refactoring	50	42	8	84% ✅
Documentation	27	19	8	70% ⚠️

Top performers: Testing (85%) and Refactoring (84%). Lowest: Documentation (70%).

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 193 words (nearly identical to closed PRs — length alone is not a predictor)
• Code/symbol references (backtick-quoted identifiers): 97% of merged PRs include them
• File/path references: 88% of merged PRs reference specific files or packages
• Most common action verbs/keywords in merged bodies: fix, consolidate, normalize, extend, enforce, resolv

Example successful prompts:

1. PR Skillet: fix sparse checkout path typing in pre-activation skills checkout #40684 — Skillet: fix sparse checkout path typing in pre-activation skills checkout
   Backtick-anchored to specific component + exact failure mode (git exit 128) → Merged

2. PR refactor: consolidate 6 identical hasStringKey copies into pkg/setutil.Contains #40534 — refactor: consolidate 6 identical hasStringKey copies into pkg/setutil.Contains
   Quantified scope (6 copies), named target package, zero ambiguity → Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 195 words (similar to merged — verbosity alone does not explain closure)
• Code references: 92% (lower than merged by ~5 points)
• WIP/Draft titles: 59% of WIP PRs are closed vs ~20% overall — strongest predictor
• Common failure modes: niche schema enrichment, multi-phase refactors with staged framing, purely additive docs with no measurable gap

Example unsuccessful prompts:

1. PR Add VideoObject JSON-LD to docs video embeds #40677 — Add VideoObject JSON-LD to docs video embeds
   Additive schema markup without a concrete problem statement → Closed

2. PR refactor(workflow): Phase 1 function-length reduction — top 5 offenders under 60 lines #39953 — refactor(workflow): Phase 1 function-length reduction — top 5 offenders under 60 lines
   "Phase 1" framing signals incomplete scope → Closed

Key Insights

• WIP/Draft titles are the strongest closure predictor: 59% of WIP PRs are closed vs ~20% overall. Don't open a PR until the change is ready.
• Code references are near-universal in successes: 97% of merged PRs use backtick-quoted identifiers. Closed PRs dip to 92% — concrete anchoring to symbols and files consistently differentiates successful prompts.
• Documentation PRs face the toughest review bar (70% success): PRs that add schema, JSON-LD, or purely additive content without anchoring to a measurable gap (audit score, missing coverage, broken behavior) are disproportionately closed.
• Length is a red herring: Merged (193 words) and closed (195 words) are nearly identical. Specificity and code anchoring matter far more than word count.
• Multi-phase staging underperforms: "Phase 1" or staged refactors have ~50% success. Single-scope, self-contained PRs are consistently stronger.

Recommendations

1. DO — Anchor every prompt to a concrete, observable failure: include the exact error message, failing test, or audit score that motivates the change.
2. DO — Reference specific symbols, files, and packages using backtick notation. 97% of merged PRs do this.
3. DO — Write single-scope, self-contained PRs. If the problem is large, pick the highest-value slice rather than framing it as "Phase 1".
4. AVOID — WIP/Draft titles: they predict a 59% closure rate.
5. AVOID — Documentation-only PRs that add content without a measurable gap (missing schema, broken link, failing audit score).

Historical Trends

Last 7 days success rates

Date	PRs	Merged	Closed	Success Rate
2026-06-21	1,000	795	202	79.5%
2026-06-20	1,000	792	204	79.2%
2026-06-19	1,000	795	201	79.5%
2026-06-18	1,000	792	196	79.2%
2026-06-16	1,000	797	196	79.7%
2026-06-14	1,000	791	202	79.1%
2026-06-13	1,000	792	204	79.2%

Trend: Success rate is remarkably stable at 79–80% across all 10 historical data points (Jun 8–21). No significant drift detected. The slight peak on Jun 16 (79.7%) was not sustained. Category-level stability is also consistent — testing and refactoring have led throughout.

References:
§27917299118

Generated by 🔍 Copilot PR Prompt Pattern Analysis · 62.8 AIC · ⌖ 16 AIC · ⊞ 8K · ◷

• expires on Jun 22, 2026, 1:08 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-06-22T21:08:38.908Z.

Closed by Workflow