Skip to content

Free-plan 20-job cap: PR Auto-Review Ready Check has no concurrency and fans out per check_suite/workflow_run — dedupe the default-branch-context runs only #1126

Closed
#1131
Closed
Free-plan 20-job cap: PR Auto-Review Ready Check has no concurrency and fans out per check_suite/workflow_run — dedupe the default-branch-context runs only#1126
#1131
Labels
bugBug reportsdev-leadFor dev-lead agent pickup

Description

@don-petry
don-petry
opened on Sep 13, 2026

Summary

The org is on GitHub Free: 20 concurrent jobs, org-wide. On 2026-09-13 at 04:40Z the two infra repos created ~250 workflow runs in 20 minutes, and Actions stalled: .github-private had 38 runs queued and 0 running, .github had 32 queued and 2 running. A gated canary-rollout promote sat in queued for over 10 minutes behind it. Actions was operational (githubstatus.com), and billing net was $0, so this is the job cap, not an outage or spending limit.

PR Auto-Review — Ready Check is the largest fan-out source that can be deduplicated safely. Its caller stub has no concurrency: at all, and it fires on four events.

Evidence: runs created 04:40–05:00Z

workflow (.github-private) runs cancelled skipped uses a runner?
Dev-Lead Agent 59 26 1 yes; tracked in .github-private#1741 / PR #1742
PR Review — Mention Trigger 34 0 34 no (runner=<none>)
Persona — Mention Router 32 0 32 no
PR Auto-Review — Ready Check 31 0 0 yes, every run
PR Review Agent — Stuck-Review Sweep 29 20 0 yes
PR Review Agent — Trigger 26 11 5 yes

Job-if:-skipped runs get no runner and complete instantly, so they are noise, not cap pressure. They are out of scope.

Why only part of Ready Check can be cancelled

Ready Check runs split by trigger context (sample, 04:40Z onward):

trigger runs head_branch check attaches to the PR head?
check_suite 19 main no, default-branch context
workflow_run 7 main no, default-branch context
pull_request 5 PR branch yes
pull_request_review 14 PR branch yes

Queued PR-context runs already show on the PR head: PR .github-private#1793 at 7881aa6e carries two queued pr-auto-review / check-and-dispatch check runs. A cancelled or evicted PR-context run would leave a cancelled check on the head, which is the .github-private#1422 / #1427 stranding shape. Default-branch-context runs cannot paint the PR, so superseding them is safe.

Acceptance Criteria

  1. Add a workflow-level concurrency: to the Ready Check caller that only deduplicates default-branch-context triggers:
    • For check_suite and workflow_run: group keyed on the workflow plus the PR number (github.event.check_suite.pull_requests[0].number / github.event.workflow_run.pull_requests[0].number), with cancel-in-progress: true.
    • For pull_request and pull_request_review: a group that is unique per run (e.g. includes github.run_id), so those runs behave exactly as today and never produce a cancelled check on a PR head.
    • cancel-in-progress as an expression that is true only for the two default-branch-context events.
    • If pull_requests[0] is empty (fork, or no associated PR), fall back to a unique-per-run group. Never collapse unrelated PRs into one group.
  2. Change the template and every live copy together: standards/workflows/pr-auto-review.yml, petry-projects/.github/.github/workflows/pr-auto-review.yml, and the .github-private copy. Both live copies currently differ from the template; reconcile that drift in the same change, or the next resync reverts it. The stub header's MUST NOT change list (trigger events, job-level permissions:) is untouched by this.
  3. Test: for each of the four events, the resolved group and cancel-in-progress are as specified (a yq/expression assertion is fine), and a check_suite event with no associated PR yields a unique group.
  4. Observed outcome after deploy, recorded on this issue body: over a comparable burst, Ready Check check_suite + workflow_run runs per PR per head SHA drop to about one completed run, and no PR shows a cancelled pr-auto-review / check-and-dispatch check.
  5. shellcheck/actionlint clean where applicable; tests registered in the existing job.

Out of scope, tracked elsewhere

  • Dev-Lead Agent cancellations: .github-private#1741 (PR #1742).
  • The Stuck-Review Sweep group is keyed per workflow_run.head_branch even though a sweep scans every PR, so bursts start one sweep per branch (29 runs, 20 cancelled). Worth a follow-up if this lands cleanly.
  • The Free-plan 20-job cap itself is an owner decision (plan/runners), not code.

Release note

Filed during the queue storm and deliberately not released yet. Releasing work adds runs to a saturated queue. Release once the org queue drains.

Activity

  1. don-petry
    added
    bugBug reports
    dev-leadFor dev-lead agent pickup
    on Sep 13, 2026

  2. don-petry commented on Sep 13, 2026

    @don-petry
    don-petry
    on Sep 13, 2026
    ContributorAuthor

    Dev-Lead Implementation Plan

    Issue: #1126 — Free-plan 20-job cap: PR Auto-Review Ready Check has no concurrency and fans out per check_suite/workflow_run — dedupe the default-branch-context runs only

    Scope

    Add a workflow-level concurrency: to the PR Auto-Review — Ready Check caller stub that deduplicates ONLY the default-branch-context triggers (check_suite / workflow_run) keyed on the PR number, while leaving pull_request / pull_request_review runs unique-per-run so they never leave a cancelled check on a PR head.

    Implementation Checklist

    • standards/workflows/pr-auto-review.yml — add concurrency: block (template / source of truth)
    • .github/workflows/pr-auto-review.yml — add identical concurrency: block (this repo's live copy); reconcile drift so the next resync does not revert it
    • .github-private/.github/workflows/pr-auto-review.yml — same block (cross-repo; noted in report — cannot be edited from this repo's workspace)
    • Verify the caller-stub surface-drift guard (scripts/compliance-audit.sh stub_surface_drift) stays clean because template and live copy carry byte-identical concurrency: blocks
    • MUST NOT change surfaces (trigger events, job-level permissions:) left untouched

    Concurrency design

    • check_suite / workflow_run: group = pr-auto-review-ready-check-pr-<PR#> from github.event.<event>.pull_requests[0].number, shared across both events for the same PR, cancel-in-progress: true.
    • pull_request / pull_request_review: group = pr-auto-review-ready-check-unique-<github.run_id> (unique per run), cancel-in-progress: false.
    • Empty pull_requests[0] (fork / no PR): falls back to the unique-per-run group so unrelated PRs never collapse.
    • cancel-in-progress is an expression that is true only for check_suite / workflow_run.

    Tests to Write

    • test/workflows/pr-auto-review/concurrency.bats — for each of the four events assert the resolved group and cancel-in-progress; assert a check_suite event with no associated PR yields a unique group; assert two different PRs never share a group; assert template and live copy carry identical blocks.
    • Registered by the existing PR Auto-Review Tests workflow (globs already cover standards/workflows/pr-auto-review.yml and test/workflows/pr-auto-review/**).

    Test Command

    bats --print-output-on-failure test/workflows/pr-auto-review/ and bash .dev-lead/scripts/dev-lead-lint.sh

  3. don-petry commented on Sep 13, 2026

    @don-petry
    don-petry
    on Sep 13, 2026
    ContributorAuthor

    Dev-Lead: Progress (in progress — not yet durable)

    Provisional. Nothing is pushed yet; the automation posts the authoritative
    completion record (with PR number + head SHA) after the push. Treat the notes
    below as intent, not delivered results.

    Plan Execution (intended)

    • Added a workflow-level concurrency: block to the Ready Check caller stub that dedupes ONLY default-branch-context triggers.
    • group: check_suite / workflow_run collapse onto one group per PR — pr-auto-review-ready-check-pr-<number> from github.event.<event>.pull_requests[0].number — with cancel-in-progress: true. pull_request / pull_request_review (and any default-branch trigger with an empty pull_requests[0]) fall through to a unique-per-run group pr-auto-review-ready-check-unique-<github.run_id> and never cancel, so no cancelled pr-auto-review / check-and-dispatch check can land on a PR head.
    • cancel-in-progress: expression that is true only for check_suite / workflow_run.
    • Wrote test/workflows/pr-auto-review/concurrency.bats (resolved group + cancel per event, the no-associated-PR check_suite → unique-group case, and template↔live-copy parity). It runs under the existing PR Auto-Review Tests job (globs already cover the stub and the test dir).

    Files touched (this workspace, not yet pushed)

    • standards/workflows/pr-auto-review.yml: added the concurrency: block (template / source of truth).
    • .github/workflows/pr-auto-review.yml: added the byte-identical concurrency: block (this repo's live copy) so a standards resync will not revert it and the caller-stub surface-drift guard stays clean.
    • test/workflows/pr-auto-review/concurrency.bats: new test file.

    Local checks (provisional — NOT the pushed run)

    • bats test/workflows/pr-auto-review/ — reports green locally.
    • bash .dev-lead/scripts/dev-lead-lint.sh — reports green locally.
    • actionlint on both stubs — reports clean locally.
    • test/scripts/compliance-audit/stub-surface-drift.bats — reports green locally (template and live copy carry identical concurrency: blocks).

    Notes / follow-ups

    • .github-private copy (AC Add multi-agent isolation strategy using git worktrees #2, third file): lives in a separate repo and is not reachable from this workspace, so it is not modified here. It still needs the same block applied (the deploy sweep re-pins the uses: line at .github-private's computed tier); flagging so it is not missed.
    • AC Add workflow, environment, and orchestration guidance #4 (observed outcome after deploy): a post-deploy measurement to be recorded on this issue once a comparable burst occurs; it cannot be produced pre-push. Per the release note, releasing is deliberately held until the org queue drains.
    • The MUST NOT change surfaces (trigger events, job-level permissions:) are untouched.

  4. don-petry commented on Sep 13, 2026

    @don-petry
    don-petry
    on Sep 13, 2026
    ContributorAuthor

    Dev-Lead: Implementation Complete — PR #1131

    The implementation for issue #1126 is durable: commits are pushed and a pull request is open. This record is backed by verifiable artifacts you can confirm in one click.

    Review happens on the PR. Acceptance criteria and test results are evidenced by the PR's diff and CI checks — not asserted here. If a later stage fails, this claim is retracted in place.

  5. don-petry
    added a commit that references this issue on Sep 16, 2026

    feat: implement issue #1126 — Free-plan 20-job cap: PR Auto-Review Re…

    6b66494
  6. don-petry
    closed this as completedin #1131on Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug reportsdev-leadFor dev-lead agent pickup

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions