Handle Copilot Review Effort Levels - #876
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the scripts/pr_review.py review automation to treat missing copilot_work_started telemetry as inconclusive for effort-labeled Copilot reviews, while also reporting the effective review effort level (Lite/Balanced/Max) from completed review metadata.
Changes:
- Remove the “request not picked up” stall classification that depended on
copilot_work_started, and keep pending requests asPENDINGuntil timeout. - Parse and report effective review effort (
effort/effort_source) in the digest without affecting coverage or completion verdicts. - Update contracts and tests across the runbook/docs/skills to match the new behavior, while keeping deprecated flags/options compatible.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/pr_review.py | Removes pickup-stall logic, adds effort parsing/reporting, keeps deprecated options compatible. |
| scripts/tests/test_pr_review.py | Replaces pickup/stall tests with effort metadata and pending-without-pickup coverage. |
| scripts/README.md | Updates user-facing behavior docs for pending/stall semantics and effort reporting. |
| .github/skills/pr-review-conduct/SKILL.md | Documents effort reporting and the “missing pickup is inconclusive” rule. |
| .agents/skills/pr-review-conduct/SKILL.md | Mirrors the pr-review-conduct skill update for agents distribution. |
| .claude-plugin/fleet-skills/skills/pr-review-conduct/SKILL.md | Mirrors the pr-review-conduct skill update for plugin distribution. |
| .github/copilot-instructions.md | Adds runbook guidance about effort observation and pickup inconclusiveness. |
| .claude-plugin/fleet-skills/.source-digest | Updates the plugin source digest to reflect skill content changes. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
ptr727
added a commit
that referenced
this pull request
Aug 20, 2026
## Summary - Treat missing `copilot_work_started` as inconclusive for effort-labeled reviews. - Observe effective Lite, Balanced, or Max metadata without changing user settings. - Keep review guidance, generated skills, and regression coverage aligned. ## Verification - Feature PR #876 completed full Copilot review on its current head. - 730 Python tests passed after review fixes. - Repository-wide ruff and mypy passed. - Distribution, prose, repository, spec, and diff gates passed. - Docker editorconfig-checker, ShellCheck, and PSScriptAnalyzer passed. Closes #868.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
copilot_work_startedas inconclusive for effort-labeled reviews.Verification
Related to #868.