Skip to content

fix(source-control): close the docs-only evidence gaps Security & Cost found after #8791 merged #8827

Closed

Description

@kyle-sexton

Problem

#8791 (security-lane docs-only skip as review evidence) merged at 2026-10-21 04:34Z on the owner's auto-merge. Security & Cost put it on HOLD at f143a26 after the merge armed, and poteto withdrew his OK. The gaps are now on main (plugins/source-control/skills/babysit-prs/scripts/babysit_review_evidence.py):

  1. The stale-marker check (~:247-253) takes min(created_at) over every listed run. It still binds when the reviewing run was deleted and the list holds (C) an earlier draft-skipped run, (D) an earlier rate-limit-failed run with no record, (E) skip, deleted review, skip, or (F) a run with no created_at (the min becomes "" and the check never fires).
  2. The docs allowlist (~:75-81) still counts docs/plugin-philosophy.md (AGENTS.md cites it for code-review rules), docs/upstream/** and docs/architecture/** as docs.
  3. P5: INSTRUCTIONS_RE lacks AGENTS.override, which HIGH_FLOOR covers.

Exploiting 1 needs Actions write (deleting a run). Measured reach of the bind: 0 of the last 100 merged lane PRs would have qualified.

Fix options (S&C)

  • 1: take the min only over runs with a completion record on a completed attempt; unbind on a missing or unparseable created_at; better, require the marker inside a recorded attempt's started_at..completed_at window. (E) needs an append-only per-review record, or refusing the docs-only bind when the lane left any other comment or thread.
  • 2: an explicit allowlist (README/CHANGELOG outside docs/, named docs/ files, deny the rest of docs/), or derive the deny set from what AGENTS.md and REVIEW.md cite.
  • Or revert feat(source-control): count the security lane's docs-only skip as review evidence for all-docs pull requests #8791: it would have covered no recent lane PR. That is the owner's call (his close-or-widen question was still open).

poteto reviews, S&C re-runs at the fix head, the owner merges (merge gate). Probes: S&C session scratchpad probe/.

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 21, 2026
  2. github-actions commented on Oct 21, 2026

    @github-actions
    Contributor

    Thanks for the report. Automated triage handed this issue to a maintainer.

    Proposed work class: work-class: structural
    Basis: automated intake triage; a maintainer confirms or changes it.

    Automated triage. A maintainer reviews it.

  3. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 21, 2026
  4. kyle-sexton commented on Oct 21, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded. Owner, 2026-10-21 ("Lets go w/ recommendations"): revert #8791, as Security & Cost and poteto recommended. As built it bound 0 of the last 100 lane PRs, so reverting costs nothing and removes the deleted-run hiding and the docs/ path bypasses. A docs-only evidence rule comes back only if a real lane needs it, with the recorded-attempt window check and an explicit docs allowlist, and Security & Cost passes it again.

  5. added a commit that references this issue on Oct 21, 2026
    71c1d5a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions