Skip to content

dev-lead claim retraction does not cover issue-comment dispositions: a fixed reply for a commit that never landed stays up #2156

Closed

Description

@don-petry

Problem

When a dev-lead pass answers a PR issue comment it posts a reply marked disposition=fixed that cites a commit. If that commit never lands, the reply stays up saying "fixed". The claim-retraction sweep from #2013 does not look at these replies: retract_unlanded_claims (scripts/dev-lead-fix-reviews.sh) lists only pulls/{pr}/comments, the review-thread replies.

The verified-disposition minimizer (#1813) does check the cited SHA, so the original comment is not wrongly resolved. The gap is the reply's text: a person or a review bot reading the PR sees "fixed" for a change that is not on the branch.

Evidence

PR #2080, 2026-10-09 at 02:42Z: a pass posted a review-thread claim (comment 4226119528) and an issue-comment disposition marked disposition=fixed (comment 6073169970), both citing 9313650. That commit never reached the branch: an auto-rebase merge of main had landed at about 02:40Z and the runs on the branch were cancelled. The review-thread claim was retracted about 15 minutes later by a later pass. Nothing retracts the issue-comment reply.

Acceptance criteria

  1. Issue-comment fixed dispositions are swept. A disposition=fixed reply posted as a PR issue comment whose cited commit is not on the remote head is retracted the same way as a review-thread claim: the disposition marker is removed, a retraction notice is added, and the original text is kept as a quote.
  2. Both sweeps cover it. It applies to the end-of-pass sweep (retract_unlanded_claims) and to the start-of-pass sweep of earlier passes added for dev-lead claim retraction: replies from cancelled runs are never swept, and a clean rebase retracts claims for fixes that landed (follow-up to #2013) #2032.
  3. Other dispositions are untouched. invalid, answered, informational and out-of-scope replies cite no commit and are never retracted by this.
  4. Tests. bats, with a gh stub: a fixed issue-comment reply citing an unlanded commit is retracted; one citing a landed commit is kept; a non-fixed disposition is left alone.

Sequencing

Build this on the sweep functions from #2032 ("dev-lead claim retraction: replies from cancelled runs are never swept, and a clean rebase retracts claims for fixes that landed"). It is blocked by that issue.

Activity

  1. github-actions commented on Oct 9, 2026

    @github-actions
    Contributor

    Thanks for the thorough report, and for including the PR #2080 evidence with the comment IDs. This looks like a clear gap: retract_unlanded_claims only lists pulls/{pr}/comments, so disposition=fixed replies posted as issue comments are never swept. One thing to confirm before work starts: since this is blocked by #2032, should issue-comment replies be identified purely by the disposition=fixed marker plus a cited SHA, or is there another marker the sweep should also match to avoid retracting replies written by humans?

  2. don-petry commented on Oct 10, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #2156 — dev-lead claim retraction does not cover issue-comment dispositions: a fixed reply for a commit that never landed stays up

    Scope

    Extend both claim sweeps (end-of-pass retract_unlanded_claims and start-of-pass sweep_earlier_claims) to also list the PR's issue comments and retract our disposition=fixed replies whose cited commit is not on the remote head.

    Implementation Checklist

    • scripts/lib/claim-landing.sh: pure cl_select_fixed_dispositions <comments_json> <bot_user> <since_iso> <pass|earlier> (our disposition=fixed issue-comment replies in the time window → <id>\t<sha>; a malformed fixed marker yields an empty sha = unverifiable) and cl_retract_disposition_body <body> <reason> (strips every disposition marker, prepends a retraction notice, quotes the original, appends <!-- dev-lead:retracted reason=… -->). Other dispositions are never selected.
    • scripts/dev-lead-fix-reviews.sh: new sweep_fixed_dispositions <ref> <pass|earlier> that lists issues/{pr}/comments, checks each cited SHA against the remote head, and PATCHes issues/comments/{id}. In the end-of-pass mode a SHA the push guard rebased is re-pointed at its landed successor instead of retracted (as for thread claims, dev-lead claim retraction: replies from cancelled runs are never swept, and a clean rebase retracts claims for fixes that landed (follow-up to #2013) #2032); in the start-of-pass mode a commit GitHub knows (pushed then rewritten) or one missing from a shallow checkout is left alone, as for thread claims.
    • Call it from retract_unlanded_claims and sweep_earlier_claims (failures fold into their return code).
    • Source comment-disposition-verify.sh before claim-landing.sh so the single disposition parser is reused.

    Tests to Write

    • tests/dev-lead/unit/test_claim_sweep.bats: earlier fixed issue-comment reply citing an unlanded commit is retracted (marker removed, notice added, original quoted); one citing a landed commit is kept; invalid/answered/informational/out-of-scope replies are untouched; end-of-pass sweep retracts this pass's unlanded fixed reply and keeps a landed one.
    • tests/dev-lead/unit/test_claim_landing.bats: unit tests for the two pure helpers (selection window, non-fixed excluded, retracted body carries no disposition marker and is idempotent).

    Test Command

    bats tests/dev-lead/unit/test_claim_sweep.bats tests/dev-lead/unit/test_claim_landing.bats (full: bats -r tests/)

  3. don-petry commented on Oct 10, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete — PR #2231

    The implementation for issue #2156 is durable: commits are pushed and a pull request is open. This record is backed by verifiable artifacts you can confirm in one click.

    Review happens on the PR. Acceptance criteria and test results are evidenced by the PR's diff and CI checks — not asserted here. If a later stage fails, this claim is retracted in place.

  4. added a commit that references this issue on Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug reportsdev-leadFor dev-lead agent pickup

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions