Skip to content

issue-triage applies needs-human-review as a blanket hold on every bug report — well-specified issues are parked until a human clears the label by hand #1778

Closed
#1783
Closed
issue-triage applies needs-human-review as a blanket hold on every bug report — well-specified issues are parked until a human clears the label by hand#1778
#1783
Labels
bugBug reportsdev-leadFor dev-lead agent pickupenhancementFeature requests

Description

@don-petry
don-petry
opened on Sep 13, 2026

Summary

issue-triage applies needs-human-review to every bug report, and hold-gate.sh treats that
label as a fleet hold that blocks dev-lead, the initiative driver, and auto-release. So filing a
well-specified bug with written acceptance criteria silently parks it until a human notices and
removes the label by hand.

The triage rule is blanket (.github/workflows/issue-triage.md):

"Add needs-human-review for bugs and ambiguous reports that need human review."

In practice the "and ambiguous reports" qualifier does nothing — bug alone is sufficient, so
every issue labelled bug acquires a hold.

Observed, repeatedly

Five issues filed in this workstream, all with explicit acceptance criteria, all auto-held within
minutes of creation:

Issue Filed Held at Gap
#1734 11:38:15Z 11:45:18Z 7 min
#1698 — — held
#1665 — — held
#1775 02:4x within minutes held
#1776 02:4x within minutes held

Each required a manual label removal before any agent would touch it. #1775 and #1776 were held
again immediately after being filed as splits of an already-triaged issue.

Why this is a defect and not a preference

The hold is load-bearing and correct — the application is what is wrong. #1595 deliberately made
needs-human-review gate the initiative driver after #1532 shipped with the hold still attached, and
scripts/lib/hold-gate.sh centralised the check so the hold set can never drift between callers.
That design is right. The problem is upstream: a hold that means "an agent must not proceed until a
human decides something" is being applied to items where nothing is awaiting a decision.

The label's own description disagrees with how triage uses it:

needs-human-review — "Flagged by automated PR review agent"

It is a PR-review escalation label — applied by the budget code in
scripts/lib/pr-automation-budget.sh when a PR exhausts its automation cycles. Triage reusing the
same label for "this is a bug report" overloads one label with two unrelated meanings, and the hold
semantics from the first leak onto the second.

It also degrades the signal. If every bug carries the hold, the label stops distinguishing items
that genuinely need a human. #1532 shipped with the hold attached precisely because the hold was
not meaningful at the release point.

Relationship to #1767 (closed)

#1767 fixed the visibility half — dev-lead now announces when it withholds action and names the
blocking label, so a held item no longer looks like a stalled agent. That was the right fix for the
symptom it addressed, and its evidence (PR #1742 sitting four days while three restart attempts were
made) is the same cost pattern seen here.

This issue is the upstream half: reducing the number of items that acquire a hold they never
needed. The two are complementary — #1767 makes holds legible, this makes them meaningful.

Acceptance Criteria

  1. Triage stops applying a hold label to routine bug reports. Either:

    • it applies needs-human-review only when the report is genuinely ambiguous or lacks enough
      information to act on (make the qualifier operative, with stated criteria); or
    • it uses a distinct non-hold label for "triaged as a bug" and leaves needs-human-review to the
      PR-review escalation path that owns it per the label description.

    State which path was taken and why.

  2. The chosen behaviour is verifiable, not just prompt wording: a test or fixture asserting that a
    well-specified bug report does not acquire a hold label. Prompt-only rules drift.

  3. The hold semantics are not weakened. hold-gate.sh keeps treating needs-human-review as a
    hold wherever it is genuinely applied — needs-human-review does not gate the Initiative Driver — #1532 was auto-released and merged with the hold label still attached #1595's gating and conflict-integrity parser misses next-line-brace function definitions — latent blind spot in both the #1482 detector and the #1520 gate #1532's lesson stand. This issue changes
    when the label is applied, never what it means.

  4. If the label keeps its dual role, the taxonomy documents both meanings explicitly
    (standards/github-settings.md label table) so the description no longer contradicts the usage.

  5. Existing wrongly-held open issues are swept once as part of the change, so the fix is not
    prospective-only. Report which were cleared.

Dev Notes

References

Activity

  1. don-petry
    added
    bugBug reports
    on Sep 13, 2026
  2. github-actions
    added
    enhancementFeature requests
    on Sep 13, 2026

  3. github-actions commented on Sep 13, 2026

    @github-actions
    github-actionsBot
    on Sep 13, 2026 – with GitHub Actions
    Contributor

    Thanks for filing such a thorough report — the evidence table, the label-description mismatch, and the explicit preservation of #1595's hold semantics make the problem very easy to reason about. This is clearly a defect in how the qualifier 'and ambiguous reports' is applied, and the fix has real design implications for the label taxonomy. The single most useful input before starting: do you have a preference between the two paths in AC1 — making the qualifier operative with explicit ambiguity criteria (so well-specified bugs bypass the hold inline), versus introducing a distinct non-hold label for 'triaged as bug' and reserving needs-human-review exclusively for the PR-review escalation path the label description already describes? Either satisfies the acceptance criteria; knowing which direction fits the team's taxonomy philosophy would help scope the change correctly.

  4. don-petry
    added
    dev-leadFor dev-lead agent pickup
    on Sep 13, 2026

  5. don-petry commented on Sep 13, 2026

    @don-petry
    don-petry
    on Sep 13, 2026
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #1778 — issue-triage applies needs-human-review as a blanket hold on every bug report

    Scope

    Stop issue-triage from applying the needs-human-review hold label by removing it from triage's safe-outputs.add-labels.allowed set (AC #1 — Path B), so a well-specified bug is classified (bug) but never auto-held; the hold label reverts to its sole owner, the PR-review escalation path.

    Why Path B (option 2), not option 1

    needs-human-review's own label description is "Flagged by automated PR review agent" — it is a PR-review escalation label (scripts/lib/pr-automation-budget.sh). Removing it from triage's allowed-set makes the fix enforced by aw.sh safe-output apply (the label is rejected even if the prompt wording later drifts), satisfying AC #2's "prompt-only rules drift" requirement. It also dissolves the dual-role overload, so AC #4's "if the label keeps its dual role" clause no longer applies.

    Implementation Checklist

    Tests to Write / Update

    • New guard in tests/aw/issue-triage/test_aw_run.sh: a triage result containing needs-human-review is rejected by safe-output apply (drift-proof — AC Go-live improvements for PR review agent #2)
    • New guard: needs-human-review is absent from issue-triage.md allowed list
    • Update existing Tests 6, 8, 11 (which encoded the old contract that triage applies needs-human-review) to the new contract

    Sweep (AC #5)

    Of 18 open issues carrying needs-human-review, 14 were bot-applied (triage classification holds) and 4 were human-applied (#1754, #1707, #1688, #1637 — genuine holds, preserved). Sweep the 14 bot-applied holds once and report which were cleared.

    Test Command

    bash tests/aw/issue-triage/test_aw_run.sh and bash .dev-lead/scripts/dev-lead-lint.sh

  6. don-petry commented on Sep 13, 2026

    @don-petry
    don-petry
    on Sep 13, 2026
    CollaboratorAuthor

    Dev-Lead: Progress (in progress — not yet durable)

    Provisional. The code changes below are not pushed yet — the automation posts the authoritative completion record (with PR number + head SHA) after the push. Treat the code notes as intent, not delivered results. (The label sweep, by contrast, is a live API action already applied.)

    Approach — Path B (AC #1, option 2)

    Removed needs-human-review from issue-triage.md's safe-outputs.add-labels.allowed set so triage classifies (bug/etc.) but never applies a hold. The label's own GitHub description is "Flagged by automated PR review agent" — it belongs to the PR-review escalation path (scripts/lib/pr-automation-budget.sh). Keeping it out of triage's allowed set makes the rule code-enforced by aw.sh safe-output apply (rejected even if the prompt drifts), which is the drift-proof property AC #2 asks for. scripts/lib/hold-gate.sh is untouched (AC #3).

    Files touched (this workspace, not yet pushed)

    • .github/workflows/issue-triage.md: drop needs-human-review from allowed labels; rewrite task step 2 to classification-only, no hold.
    • tests/aw/issue-triage/test_aw_run.sh: Test 8 now asserts the label is absent from the allowed set; Test 11 now asserts safe-output rejects a triage result carrying it; Test 6 valid-case fixture is bug alone.
    • tests/aw/issue-triage/scenarios.md: Scenario 1 expects bug only.

    Local check output (provisional — not the pushed run)

    tests/aw/issue-triage/test_aw_run.sh   → 12 passed, 0 failed
tests/aw/issue-triage/test_runner_yml.sh → 7 passed, 0 failed
tests/aw/validate-specs.sh             → 30 passed, 0 failed
.dev-lead/scripts/dev-lead-lint.sh     → all checks passed

    Sweep (AC #5 — live, already applied)

    Of the open issues carrying needs-human-review, the 14 bot/triage-applied holds were cleared:
    #1744 #1743 #1740 #1733 #1687 #1620 #1592 #1568 #1566 #1495 #1318 #1198 #1160 #865.
    3 human-applied holds were preserved as genuine (#1707, #1688, #1637), keying the sweep on the last labeled event's actor type (Bot vs User) so a deliberate human hold is never undone. Label removal is reversible if any needs re-holding.

    Notes

    • AC Optimize review: small-PR and incremental fast paths #4: the label no longer has a dual role (triage stopped using it), so its description now matches its sole owner; standards/github-settings.md lives in the public petry-projects/.github repo, not here, so there is no local taxonomy table to amend.

  7. don-petry commented on Sep 13, 2026

    @don-petry
    don-petry
    on Sep 13, 2026
    CollaboratorAuthor

    Dev-Lead: Implementation Complete — PR #1783

    The implementation for issue #1778 is durable: commits are pushed and a pull request is open. This record is backed by verifiable artifacts you can confirm in one click.

    Review happens on the PR. Acceptance criteria and test results are evidenced by the PR's diff and CI checks — not asserted here. If a later stage fails, this claim is retracted in place.

  8. don-petry
    added a commit that references this issue on Sep 19, 2026

    feat: implement issue #1778 — issue-triage applies needs-human-review…

    8761759
  9. don-petry
    closed this as completedin #1783on Sep 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug reportsdev-leadFor dev-lead agent pickupenhancementFeature requests

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions