Skip to content

Enable GitHub pull request limits to cut automation-PR noise #505

Closed
7 of 7 issues completed
Closed
Enable GitHub pull request limits to cut automation-PR noise#505
7 of 7 issues completed
Labels
initiativeinitiative:autoArms an initiative epic for the initiative-driver auto-release

Description

@don-petry
don-petry
opened on Jun 21, 2026

Why

Discussion #497 proposes adopting GitHub's pull request limits to cut down the review/CI noise generated by this org's heavy PR automation. Today the only PR cap we run is Dependabot's ecosystem-scoped open-pull-requests-limit: 10 (.github/dependabot.yml); there is no repo- or org-wide ceiling on concurrent open PRs, even though dev-lead, auto-rebase fan-out, and the initiative-driver all open PRs continuously.

What

Research the exact GitHub mechanism (repo setting vs. org setting vs. ruleset rule + its API), encode the limit as settings-as-code in our existing apply mechanism (scripts/apply-repo-settings.sh and/or .github/rulesets/), wire an idempotent apply path that actually enforces it (neither the script nor the ruleset JSON is invoked by any workflow today), document the standard + exemptions, and instrument the noise-reduction metric.

Success metric

Within two weeks of enforcement on the in-scope repo(s): the count of open, non-draft automation-authored PRs per repo stays at or below the configured cap, AND zero legitimate (human or sanctioned-automation) PRs are wrongly refused by the limit. Baseline and post-rollout numbers are measured by the Story 5 report.

Cost / budget bound

This is configuration + CI work, not an LLM runtime loop. Bounded to the ~5 dev-lead story PRs below. The only recurring runtime cost is a single no-LLM, gh-API + jq report capped at <=1 run/day on github.token (Story 5) - no recurring token/LLM spend.

Scope note

Final numeric limit, exempt actors, and rollout breadth (pilot repo vs. org-wide) are human decisions - see open questions; the config is built parameterized so those values are set without a code change.

Untracked prerequisites

  • Confirm GitHub 'pull request limits' is generally available on the petry-projects org plan (not gated behind a preview/enterprise-only flag) and whether it exposes a REST/GraphQL API or is UI-only. External GitHub-feature dependency, not a repo issue; resolved and recorded in the Story 1 ADR. Source: https://github.blog/open-source/maintainers/how-pull-request-limits-are-cutting-down-the-noise/
  • Human sign-off on the final numeric limit, the exempt-actor list, and the rollout scope before Story 3 flips enforcement on a live repo.

Planned from idea discussion #497 by the BMAD Scrum Master initiative-planner. Inert until a maintainer adds initiative:auto.

Activity

  1. don-petry
    added
    initiative
    on Jun 21, 2026
  2. don-petry
    added sub-issues on Jun 21, 2026
  3. don-petry
    added a commit that references this issue on Jun 25, 2026

    feat: enroll initiative-driver stub on .github (#888 pilot setup) (#530)

    3e0105e
  4. don-petry
    added
    initiative:autoArms an initiative epic for the initiative-driver auto-release
    on Jun 27, 2026

  5. don-petry commented on Jun 27, 2026

    @don-petry
    don-petry
    on Jun 27, 2026
    ContributorAuthor

    ✅ Fleet-enablement pilot: initiative-driver auto-released a story cross-repo

    Result: PASS — the central initiative-driver auto-released a not-previously-labeled sub-issue cross-repo with zero manual dev-lead labeling.

    What happened (all UTC, 2026-06-27)

    1. 13:39:10 — armed this epic: initiative:auto added to Enable GitHub pull request limits to cut automation-PR noise #505 (the only manual mutation; on the epic, not on any story). Note: the initiative:auto gate label did not yet exist on this repo (adoption checklist step 2), so it was created first, then applied.
    2. 13:39:12 — per-repo stub Initiative Driver — Dispatch Central ran on this repo from the labeled event and dispatched the central driver with target_repo=petry-projects/.github. Run: https://github.com/petry-projects/.github/actions/runs/28290875147 (success).
    3. 13:39:17→13:39:30 — central initiative-driver ran in petry-projects/.github-private (workflow_dispatch, success): https://github.com/petry-projects/.github-private/actions/runs/28290877235
    4. 13:39:26 — central driver applied dev-lead to [Phase 2a] Encode the PR-limit config as a machine-readable single source of truth #507 cross-repo via GH_PAT_WORKFLOWS — inside the driver's 13s window. [Phase 2a] Encode the PR-limit config as a machine-readable single source of truth #507 was NOT hand-labeled (its blocker [Phase 1] ADR: pull-request-limits mechanism + policy decision #506 was already closed, making it the ready story). [Phase 3] Apply path: enforce the limit on target repo(s) + tests #508/[Phase 3] Document the PR-limits standard + exemption rationale #509/[Phase 4] Measure the noise reduction (success-metric instrumentation) #510 correctly stayed unreleased (still blocked_by open issues).
    5. 13:39:28 — Dev-Lead Agent started building [Phase 2a] Encode the PR-limit config as a machine-readable single source of truth #507 in this repo (issues event from the bot label): https://github.com/petry-projects/.github/actions/runs/28290881508 (in_progress).

    Key facts

    This validates the full fleet-enabled cascade: per-repo stub → central driver dispatch → cross-repo dev-lead label → repo-local dev-lead.yml build, with no human label hand-off.

    — pilot petry-projects/.github-private#888

  6. don-petry
    added sub-issues on Jun 30, 2026
  7. don-petry
    added 3 commits that reference this issue on Jul 1, 2026

    feat(pr-limits): sign off cap at 50 as a single org-wide soft ceiling (…

    1d269ff

    feat(pr-limits): wire the admission gate into dev-lead-fix-issue.sh (#…

    ba5c874

    feat(pr-limits): add the no-LLM success-metric report (#510) (#573)

    3958d37

  8. don-petry commented on Jul 2, 2026

    @don-petry
    don-petry
    on Jul 2, 2026
    ContributorAuthor

    ✅ Initiative delivered — all stories merged

    All sub-issues are closed and the feature is on main:

    Phase Story Delivered by
    1 #506 ADR (mechanism + policy) ADR docs/initiatives/pull-request-limits-adr.md
    2a #507 machine-readable config PR #562 → standards/pr-limits.json
    2b #561 source-side admission-gate library PR #565 → scripts/lib/pr-limit-gate.sh
    — Human sign-off (gate #566) cap 50, org-wide soft ceiling — PR #570
    3 #508 apply path (enforce at source) petry-projects/.github-private #1004 → gate wired into dev-lead-fix-issue.sh
    3 #509 standard + exemption rationale PR #572 → standards/pr-limits.md + AGENTS.md
    4 #510 success-metric report PR #573 → scripts/pr-limits-report.sh + daily workflow

    Key design (per the ADR)

    GitHub exposes no native max-open-PRs surface, so enforcement is a source-side admission gate + shared config: before dev-lead/initiative automation opens a PR, it counts the open non-draft non-exempt automation queue org-wide (gh search prs) and defers if at/over the cap. Dependabot + break-glass actors + security-labeled PRs are exempt (never counted, never blocked).

    Signed-off policy

    • Org-wide soft cap = 50 (single ceiling; no per-source sub-caps). Baseline ~44 automation PRs, currently 41/50 counted → 9 headroom, so it refuses nothing today; it only bounds runaway growth.

    ⚠️ Activation status (important)

    The gate is merged to .github-private main, but the dev-lead engine runs scripts from the dev-lead/ring0 channel — so enforcement activates as this change promotes through the dev-lead channels (ring0 → stable) via the normal canary-rollout soak. It is not yet live-enforcing; riding the standard soak is safe (no-op at 41/50). The daily pr-limits-report measures the metric going forward.

    Scope note preserved

    This cap targets the open-PR-count / CI-churn cost (ADR §1/§5). The recent bot review-comment noise pain is a separate concern already addressed by the advisory-review-gate (#457/#458).

    Closing as delivered. Enforcement goes live on channel promotion; the metric is now auto-reported.

  9. don-petry
    closed this as completedon Jul 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    initiativeinitiative:autoArms an initiative epic for the initiative-driver auto-release

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions