feat: implement issue #747 — [Fleet Monitor] petry-projects/.github — .github/workflows/canary-rollout.yml - #748
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Review limit reached
Next review available in: 45 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe canary rollout helper now captures ChangesCanary workflow handling
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the _run_json function in scripts/canary-rollout.sh to prevent fail-closed errors when a workflow has never been run on a repository. It captures gh command errors in a temporary file and returns an empty array if the workflow is not found, with corresponding tests added in tests/canary_rollout.bats. The review feedback recommends using an EXIT trap to ensure the temporary file is reliably cleaned up, rather than relying on manual deletion at multiple exit points.
There was a problem hiding this comment.
Pull request overview
This PR addresses issue #747 by updating the canary rollout orchestration script’s _run_json behavior so tier repositories that don’t have a given workflow are treated as a legitimate “no data” case ([]) rather than a transient outage that triggers retries and ultimately fails the fleet sweep.
Changes:
- Update
_run_jsonto capturegh run liststderr and treat “could not find any workflows named …” as an immediate[]return (no retries, no fail-closed). - Add Bats coverage for the “workflow not present” case, including ensuring it returns immediately without retrying.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/canary_rollout.bats | Adds regression tests for _run_json when a workflow is not present in a tier repo (#747). |
| scripts/canary-rollout.sh | Adjusts _run_json to detect “workflow not present” via captured stderr and return [] instead of retry/fail-closed. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
scripts/canary-rollout.sh (1)
339-344: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick winPreserve and log the underlying error message.
By redirecting
gh's stderr to the temporary file$errf, the actual error messages (e.g., "API rate limit exceeded", "HTTP 502", "repository not found") are now swallowed and never shown in the workflow logs for genuine failures. This will make it very difficult to debug why a rollout failed.Please output the contents of
$errfbefore removing it, both on the final failure and during retries, so the root cause remains observable.🛠️ Proposed fix to log the original error
if [ "$attempt" -ge "$attempts" ]; then + cat "$errf" >&2 rm -f "$errf" echo "::error::_run_json: failed to fetch run list for $repo (workflow=$wf) after $attempts attempt(s)" >&2 return 1 fi + cat "$errf" >&2 echo "::warning::_run_json: transient failure fetching run list for $repo (workflow=$wf), attempt $attempt/$attempts — retrying in ${delay}s" >&2🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/canary-rollout.sh` around lines 339 - 344, Update the _run_json failure handling to output the contents of $errf before removing it on both the final failure path and each retry path. Preserve the existing error and warning messages while ensuring the captured gh stderr, including the underlying API or repository error, remains visible in workflow logs.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@scripts/canary-rollout.sh`:
- Around line 339-344: Update the _run_json failure handling to output the
contents of $errf before removing it on both the final failure path and each
retry path. Preserve the existing error and warning messages while ensuring the
captured gh stderr, including the underlying API or repository error, remains
visible in workflow logs.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: deb88725-646a-4f63-acde-0db2a1add666
📒 Files selected for processing (2)
scripts/canary-rollout.shtests/canary_rollout.bats
Dev-Lead — waiting on PR blockers (intent: review-changes)PR: #748 |
|
Note @don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically. |
Dev-Lead — review-changes (applied)Changes committed and pushed. |
Dev-Lead — review-changes (applied)Changes committed and pushed. |
Dev-Lead — review-changes (no-changes)No changes were needed for this PR. |
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: 50935ea21a3149cd5a9b9547e6df0c266d3d7f15
Review mode: triage-approved (single reviewer)
Summary
Test-only PR (+37 lines in tests/canary_rollout.bats) adding two regression tests for the #747 fix: _run_json must treat a tier repo that has never run the requested workflow ('could not find any workflows named …') as a legitimate empty result ([]) — immediately, without retries, and without failing the fleet sweep closed. The production fix already exists on main in scripts/canary-rollout.sh (_repo_wf_runs_cached classifies that error as permanent, caches [], and never retries); these tests lock in that behavior. Triage's low-risk assessment is confirmed.
Linked issue analysis
Issue #747 (Fleet Monitor, CRITICAL): canary-rollout.yml at 69.2% failure rate, driven by retry storms when tier repos lacked the requested workflow — each miss exhausted 6 retries with backoff, blowing sweeps out to ~20 min and getting jobs cancelled. The root fix (classify missing-workflow as empty result, no retry) is on main; this PR adds the regression coverage that pins it. Substantively addresses the issue; the tests assert both the [] result (status 0) and the single-call/no-retry behavior, while genuine fetch errors still fail closed per the adjacent existing test.
Findings
No blocking findings. Verified test correctness against the implementation on main: the stub gh emits the exact stderr string the classifier matches (case-insensitive 'could not find any workflow'), _run_json passes the empty result through jq to output [] with exit 0, and the call-log test correctly proves exactly one gh invocation with no retry warnings. Tests follow the established stub/source pattern used by neighboring tests in the same file. Secret-scanning MCP tool unavailable in this environment — relied on the green gitleaks check; diff contains only test stub content, nothing secret-like.
CI status
All quality and security checks green for 50935ea: Lint, Lint and bats, ShellCheck, CodeQL, Analyze (actions), Secret scan (gitleaks), Agent Security Scan, AgentShield, SonarCloud (quality gate passed), npm audit. The CANCELLED 'dev-lead / dispatch' and 'dev-lead / ci-relay' entries are the automation bot's own superseded runs — each has a later SUCCESS/SKIPPED rerun of the same check. mergeStateStatus BLOCKED only pending this automated review.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
Superseded by automated re-review at 50935ea.
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: 50935ea21a3149cd5a9b9547e6df0c266d3d7f15
Review mode: triage-approved (single reviewer)
Summary
Test-only PR (+37/-0 in tests/canary_rollout.bats) adding two regression tests for _run_json's workflow-not-present handling (#747): (1) a tier repo whose 'gh run list --workflow' fails with 'could not find any workflows named …' returns [] with exit 0 instead of failing closed, and (2) that path returns immediately without consuming retries. The behavior fix itself already landed on main in scripts/canary-rollout.sh (the missing-workflow message is classified as a permanent condition returning [], with only genuine transport errors retried/fail-closed), so after rebase this PR correctly carries only the regression coverage — confirmed by the green 'Lint and bats' check running these tests against main's script.
Linked issue analysis
Closes #747 — a Fleet Monitor CRITICAL alert for canary-rollout.yml (69.2% failure rate, 9/13 runs). Root cause was a retry storm: repos that never ran a given workflow caused non-zero 'gh run list' exits that were retried with backoff and then failed the whole fleet sweep. The fix on main plus these regression tests substantively address the alert; the issue is closed.
Findings
No blocking findings. The triage assessment (low-risk) is confirmed.
- Diff is test-only; stubs follow the file's existing pattern (mktemp -d stub dir prepended to PATH, CANARY_GH_RETRY_SLEEP=0). No production code touched.
- Both inline review threads (gemini-code-assist, Copilot) are resolved and outdated — they referenced earlier revisions that modified scripts/canary-rollout.sh.
- No unanswered human-reviewer questions; latest cascade review at this same SHA already approved.
- Secret scanning MCP tool not available in this environment; gitleaks CI check passed and the diff contains only test stub content — no secrets.
CI status
All validation checks green: Lint, Lint and bats, ShellCheck, CodeQL (actions), Secret scan (gitleaks), SonarCloud (quality gate passed), Agent Security Scan, agent-shield, npm audit, pr-auto-review dispatches. CANCELLED entries ('dev-lead / dispatch', 'dev-lead / ci-relay') are superseded agent-orchestration runs with later SUCCESS/SKIPPED entries, not code-quality gates. Mergeable: MERGEABLE.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
Superseded by automated re-review at 50935ea.
Dev-Lead — review-changes (no-changes)No changes were needed for this PR. |
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
codeant-ai
Bot
added
size:M
Dev-Lead — fix-bot-comment (applied)Changes committed and pushed. |
Dev-Lead — review-changes (no-changes)No changes were needed for this PR. |
|
Dev-Lead — fix-bot-comment (no-changes)Agent reasoning |
User description
Closes #747
Implemented by dev-lead agent. Please review.
Summary by CodeRabbit
Bug Fixes
Tests
CodeAnt-AI Description
Treat missing workflows as empty results during canary rollouts
What Changed
Impact
✅ Fewer failed canary fleet sweeps✅ No unnecessary retries for missing workflows✅ Clearer handling of genuine fetch failures💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.