Repository navigation
feat(dev-lead): dev-lead:hands-off label to exclude infra PRs from the agent - #454
Conversation
The agent enables auto-merge and works any PR it's triggered on, including PRs that modify the dev-lead workflow itself — so bot review comments on those infra PRs make it pile commits onto its own change (seen on broodly#292 and .github-private#451). Add an opt-out: a `dev-lead:hands-off` label on the PR or issue classifies every event as skip (reason hands-off-label), before routing. Pure payload-based label check (.pull_request.labels / .issue.labels) — no API calls, so the classifier stays side-effect-free and unit-testable. repository_dispatch payloads carry no labels and are out of scope (documented). Tests: review comment on a hands-off PR → skip; same without the label → fix-reviews (control); issue labeled dev-lead + hands-off → skip. shellcheck clean. Refs combsmith/.github#402 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 5 minutes and 1 second. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (7)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dev-Lead — review-changes (no-changes)No changes were needed for this PR. |
There was a problem hiding this comment.
Code Review
This pull request introduces a 'hands-off' guard to exclude PRs or issues labeled with 'dev-lead:hands-off' from being processed by the agent, including new test fixtures and unit tests. A critical bug was identified in the has_label function's jq filter, where the any condition is evaluated on the array itself rather than its elements, rendering the guard ineffective. A code suggestion was provided to correctly iterate and check the labels.
There was a problem hiding this comment.
Pull request overview
Adds a dev-lead:hands-off label guard to the dev-lead event classifier so infrastructure PRs (especially those modifying dev-lead itself) can be explicitly excluded from agent routing, preventing the agent from iterating on its own workflow changes.
Changes:
- Added a payload-only label detection helper (
has_label) to keep intent classification side-effect-free (no API calls). - Added a pre-routing hands-off guard that emits
skipwith reasonhands-off-labelwhen the label is present. - Added unit coverage (Bats) plus new webhook payload fixtures for labeled PR review comments and labeled issues.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
scripts/dev-lead-intent.sh |
Adds has_label helper and a hands-off pre-routing skip guard. |
tests/dev-lead/unit/test_intent_hands_off.bats |
New unit tests validating hands-off skip behavior for PR review comments and issues. |
tests/dev-lead/fixtures/events/pr_review_comment_copilot_hands_off.json |
New fixture payload for a PR review comment event with the hands-off label. |
tests/dev-lead/fixtures/events/issues_labeled_dev_lead_hands_off.json |
New fixture payload for an issues labeled event containing both dev-lead and hands-off. |
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: fd65bad345d1ae0bfde34e88144251038e866bd0
Review mode: triage-approved (single reviewer)
Summary
Adds a dev-lead:hands-off label-based opt-out to scripts/dev-lead-intent.sh so PRs/issues that modify the dev-lead workflow itself can be excluded from the agent. The check is a pure payload read (.pull_request.labels / .issue.labels), runs before routing, and emits skip with reason hands-off-label. Backed by 3 bats unit tests (hands-off PR review comment → skip; same comment without label → fix-reviews control; issue labeled dev-lead+hands-off → skip) and 2 new fixtures.
Linked issue analysis
References petry-projects/.github#402 via Refs: (not Closes). #402 is the concurrency-group root-cause investigation; this PR is an orthogonal infra-hardening change with its own independent motivation (PR body cites concrete incidents: broodly#292 and .github-private#451 where the agent piled 7 and 1 commits onto its own infra PRs). Scope and rationale are clearly stated.
Findings
- No blocking issues. Change is additive, payload-only, side-effect free.
- gemini-code-assist comment (false positive): Claimed the
jqfilter evaluatesanyagainst the wrapping array rather than its elements. This is incorrect —any(condition)in jq is sugar forany(.[]; condition), so[ (.pull_request.labels // []), (.issue.labels // []) ] | flatten | any(.name == $n)iterates each label object as intended. Confirmed by direct execution against representative payloads, and corroborated by the new bats tests passing in CI. repository_dispatchcoverage: The PR correctly notes (in code comment + PR body) thatrepository_dispatchpayloads carry no labels and are out of scope here; the right place to honor hands-off for that path is the ci-relay producer. Reasonable scope boundary.- Nit (non-blocking): Both new JSON fixtures lack a trailing newline. Cosmetic only.
CI status
All required checks green: Lint, shellcheck, bats, validate-agent-profiles, gh-aw-compile, unit-tests, CodeQL, Agent Security Scan, Secret scan (gitleaks), AgentShield, SonarCloud, dependency-audit, Compile agentic workflows. SonarCloud Quality Gate passed (0 new issues). Mergeable; merge state BLOCKED only on the standard org-leads review requirement.
Reviewed automatically by the PR-review agent (claude-opus-4-7). Reply if you need a human review.
Dev-Lead — fix-bot-comment (no-changes)Agent reasoning |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fd65bad345
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
ead3a85
Dev-Lead — rate-limited (intent: review-changes)PR: #454 |
|
Note @don-petry I received your request but all AI engines are currently rate-limited. I'll retry automatically once the rate limit clears. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ead3a85aac
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
Dev-Lead — rate-limited (intent: review-changes)PR: #454 |
|
Note @don-petry I received your request but all AI engines are currently rate-limited. I'll retry automatically once the rate limit clears. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c16ac99edf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 402a55e5d8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: 4a5ff44e964b8f76610a3a2a9fe2e12d39fb9859
Review mode: triage-approved (single reviewer)
Summary
Adds a dev-lead:hands-off label-based opt-out to scripts/dev-lead-intent.sh so PRs/issues that modify the dev-lead workflow itself can be excluded from the agent. The check runs before routing and the anti-loop guard, so a hands-off sync from BOT_USER emits hands-off-label rather than dev-lead-own-commit (stable skip reason). For label-bearing events the check is a pure payload read (.pull_request.labels / .issue.labels); for repository_dispatch (no labels in payload) it fetches labels via gh api and fails closed on lookup error. Backed by 7 bats unit tests across 3 new fixtures, with gh stubbed in test setup so the suite doesn't depend on an authenticated CLI.
Linked issue analysis
References petry-projects/.github#402 via Refs: (not Closes). #402 is the concurrency-group root-cause investigation; this PR is an orthogonal infra-hardening change with its own independent motivation. The PR body cites concrete incidents (broodly#292 and .github-private#451) where the agent piled 7 and 1 commits onto its own infra PRs, which is the bug this label addresses. Scope is clearly bounded.
Findings
- No blocking issues. Change is additive, mostly payload-only, and side-effect-free for the common path.
jqfilter is correct.any(.pull_request.labels[]?, .issue.labels[]?; .name == $n)uses jq's two-argany(generator; condition)form — the generator streams every label object from either array (with?for missing keys), and the condition checks.name. Verified by passing bats tests. The earlier gemini-code-assist comment claiming this was broken was a false positive.- Dispatch branch fails closed. New
gh apilookup for labels onrepository_dispatchevents emitsskipwith reasonlabel-lookup-erroron any non-zero exit (e.g. transient failure, token withoutissues:read). Testrepository_dispatch ci-failure with API error → skip (fail closed)covers this. Correct defensive default. - No injection surface. Label name passed via
jq --arg n, so it's data not program;gh apiURL uses the trusted$GITHUB_REPOSITORYenv var and an integer PR number parsed from the dispatch payload. - Test isolation improved.
test_intent_ci.batsandtest_intent_stub.batsnow stubghin theirsetup, so the suite doesn't silently depend on an authenticated CLI when the new dispatch branch runs. - Nit (non-blocking): Two new JSON fixtures (
issues_labeled_dev_lead_hands_off.json,pr_review_comment_copilot_hands_off.json) lack a trailing newline. Cosmetic.
CI status
All required checks green: Lint, ShellCheck, shellcheck, bats, validate-agent-profiles, gh-aw-compile, unit-tests, Compile agentic workflows, CodeQL (Analyze actions + CodeQL), Agent Security Scan, Secret scan (gitleaks), AgentShield, SonarCloud (Code Analysis + integration), dependency-audit (Detect ecosystems). SonarCloud Quality Gate passed (0 new issues, 0 hotspots, 0 duplications). Mergeable; merge state is BLOCKED only on the standard org-leads review requirement.
Reviewed automatically by the PR-review agent (single-reviewer mode: opus 4.7). Reply if you need a human review.
Dev-Lead — rate-limited (intent: fix-reviews)PR: #454 |
…e agent (#454) * feat(dev-lead): skip PRs/issues labeled dev-lead:hands-off The agent enables auto-merge and works any PR it's triggered on, including PRs that modify the dev-lead workflow itself — so bot review comments on those infra PRs make it pile commits onto its own change (seen on broodly#292 and .github-private#451). Add an opt-out: a `dev-lead:hands-off` label on the PR or issue classifies every event as skip (reason hands-off-label), before routing. Pure payload-based label check (.pull_request.labels / .issue.labels) — no API calls, so the classifier stays side-effect-free and unit-testable. repository_dispatch payloads carry no labels and are out of scope (documented). Tests: review comment on a hands-off PR → skip; same without the label → fix-reviews (control); issue labeled dev-lead + hands-off → skip. shellcheck clean. Refs combsmith/.github#402 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: Claude Code Bot <noreply@anthropic.com> Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
…e agent (#454) * feat(dev-lead): skip PRs/issues labeled dev-lead:hands-off The agent enables auto-merge and works any PR it's triggered on, including PRs that modify the dev-lead workflow itself — so bot review comments on those infra PRs make it pile commits onto its own change (seen on broodly#292 and .github-private#451). Add an opt-out: a `dev-lead:hands-off` label on the PR or issue classifies every event as skip (reason hands-off-label), before routing. Pure payload-based label check (.pull_request.labels / .issue.labels) — no API calls, so the classifier stays side-effect-free and unit-testable. repository_dispatch payloads carry no labels and are out of scope (documented). Tests: review comment on a hands-off PR → skip; same without the label → fix-reviews (control); issue labeled dev-lead + hands-off → skip. shellcheck clean. Refs combsmith/.github#402 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: Claude Code Bot <noreply@anthropic.com> Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
…e agent (#454) * feat(dev-lead): skip PRs/issues labeled dev-lead:hands-off The agent enables auto-merge and works any PR it's triggered on, including PRs that modify the dev-lead workflow itself — so bot review comments on those infra PRs make it pile commits onto its own change (seen on broodly#292 and .github-private#451). Add an opt-out: a `dev-lead:hands-off` label on the PR or issue classifies every event as skip (reason hands-off-label), before routing. Pure payload-based label check (.pull_request.labels / .issue.labels) — no API calls, so the classifier stays side-effect-free and unit-testable. repository_dispatch payloads carry no labels and are out of scope (documented). Tests: review comment on a hands-off PR → skip; same without the label → fix-reviews (control); issue labeled dev-lead + hands-off → skip. shellcheck clean. Refs combsmith/.github#402 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: Claude Code Bot <noreply@anthropic.com> Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
…e agent (#454) * feat(dev-lead): skip PRs/issues labeled dev-lead:hands-off The agent enables auto-merge and works any PR it's triggered on, including PRs that modify the dev-lead workflow itself — so bot review comments on those infra PRs make it pile commits onto its own change (seen on broodly#292 and .github-private#451). Add an opt-out: a `dev-lead:hands-off` label on the PR or issue classifies every event as skip (reason hands-off-label), before routing. Pure payload-based label check (.pull_request.labels / .issue.labels) — no API calls, so the classifier stays side-effect-free and unit-testable. repository_dispatch payloads carry no labels and are out of scope (documented). Tests: review comment on a hands-off PR → skip; same without the label → fix-reviews (control); issue labeled dev-lead + hands-off → skip. shellcheck clean. Refs combsmith/.github#402 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: Claude Code Bot <noreply@anthropic.com> Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
…e agent (#454) * feat(dev-lead): skip PRs/issues labeled dev-lead:hands-off The agent enables auto-merge and works any PR it's triggered on, including PRs that modify the dev-lead workflow itself — so bot review comments on those infra PRs make it pile commits onto its own change (seen on broodly#292 and .github-private#451). Add an opt-out: a `dev-lead:hands-off` label on the PR or issue classifies every event as skip (reason hands-off-label), before routing. Pure payload-based label check (.pull_request.labels / .issue.labels) — no API calls, so the classifier stays side-effect-free and unit-testable. repository_dispatch payloads carry no labels and are out of scope (documented). Tests: review comment on a hands-off PR → skip; same without the label → fix-reviews (control); issue labeled dev-lead + hands-off → skip. shellcheck clean. Refs combsmith/.github#402 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: Claude Code Bot <noreply@anthropic.com> Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
…e agent (#454) * feat(dev-lead): skip PRs/issues labeled dev-lead:hands-off The agent enables auto-merge and works any PR it's triggered on, including PRs that modify the dev-lead workflow itself — so bot review comments on those infra PRs make it pile commits onto its own change (seen on broodly#292 and .github-private#451). Add an opt-out: a `dev-lead:hands-off` label on the PR or issue classifies every event as skip (reason hands-off-label), before routing. Pure payload-based label check (.pull_request.labels / .issue.labels) — no API calls, so the classifier stays side-effect-free and unit-testable. repository_dispatch payloads carry no labels and are out of scope (documented). Tests: review comment on a hands-off PR → skip; same without the label → fix-reviews (control); issue labeled dev-lead + hands-off → skip. shellcheck clean. Refs combsmith/.github#402 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: Claude Code Bot <noreply@anthropic.com> Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
…e agent (#454) * feat(dev-lead): skip PRs/issues labeled dev-lead:hands-off The agent enables auto-merge and works any PR it's triggered on, including PRs that modify the dev-lead workflow itself — so bot review comments on those infra PRs make it pile commits onto its own change (seen on broodly#292 and .github-private#451). Add an opt-out: a `dev-lead:hands-off` label on the PR or issue classifies every event as skip (reason hands-off-label), before routing. Pure payload-based label check (.pull_request.labels / .issue.labels) — no API calls, so the classifier stays side-effect-free and unit-testable. repository_dispatch payloads carry no labels and are out of scope (documented). Tests: review comment on a hands-off PR → skip; same without the label → fix-reviews (control); issue labeled dev-lead + hands-off → skip. shellcheck clean. Refs combsmith/.github#402 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: Claude Code Bot <noreply@anthropic.com> Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
…e agent (#454) * feat(dev-lead): skip PRs/issues labeled dev-lead:hands-off The agent enables auto-merge and works any PR it's triggered on, including PRs that modify the dev-lead workflow itself — so bot review comments on those infra PRs make it pile commits onto its own change (seen on broodly#292 and .github-private#451). Add an opt-out: a `dev-lead:hands-off` label on the PR or issue classifies every event as skip (reason hands-off-label), before routing. Pure payload-based label check (.pull_request.labels / .issue.labels) — no API calls, so the classifier stays side-effect-free and unit-testable. repository_dispatch payloads carry no labels and are out of scope (documented). Tests: review comment on a hands-off PR → skip; same without the label → fix-reviews (control); issue labeled dev-lead + hands-off → skip. shellcheck clean. Refs combsmith/.github#402 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: Claude Code Bot <noreply@anthropic.com> Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>



Why
The agent enables auto-merge by default (#447) and works any PR it's triggered on — including PRs that change the dev-lead workflow itself. Bot review comments on those infra PRs make the agent pile commits onto its own change (observed on broodly#292 and .github-private#451, where it added 7 and 1 commits respectively).
What
A
dev-lead:hands-offlabel on the PR or issue makesdev-lead-intent.shclassify every event asskip(reasonhands-off-label), evaluated before routing — so it covers PR opens/syncs, reviews, review comments, PR issue-comments, and issue labeling..pull_request.labels/.issue.labels) — no API calls, classifier stays side-effect-free and unit-testable.repository_dispatchpayloads carry no labels → out of scope (noted in code; the ci-relay producer is where that'd belong).Tests
tests/dev-lead/unit/test_intent_hands_off.bats(3 cases): hands-off PR review comment → skip; same comment without the label →fix-reviews(control); issue labeleddev-lead+hands-off→ skip. Full intent suite still green; shellcheck clean.Follow-up
I'll apply
dev-lead:hands-offto the open dev-lead infra PRs once adev-lead:hands-offlabel exists in each repo.Refs combsmith/.github#402
🤖 Generated with Claude Code