feat(bugs): measure hunt recall before lowering bugs:scan hunter effort #6909

Activity

1. github-actions

   added

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 11, 2026
2. github-actions commented on Oct 11, 2026

   

   github-actionsBot

   on Oct 11, 2026 – with GitHub Actions

   Contributor

   More actions

   Thanks for the report. Automated triage handed this issue to a maintainer.

   _{Automated triage. A maintainer reviews it.}
3. github-actions

   added

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   on Oct 11, 2026
4. kyle-sexton commented on Oct 13, 2026

   

   kyle-sexton

   on Oct 13, 2026

   MemberAuthor

   More actions

   Decision recorded. The owner approved applying independently verified recommendations on 2026-10-12.

   • Build the seeded-bug recall eval now, or keep hunters at session effort and park the cut? → Keep hunters at session effort; close until cost data shows hunters dominate bugs:scan spend
     Basis: plugins/bugs/skills/scan/SKILL.md:155 (hunters pass no effort); AGENTS.md Code Review Rules (cut ships only with eval evidence); judgment on value

   Wants a seeded-bug recall eval for bugs:scan so hunters could run at low effort, since cuts need eval evidence of no loss. Main keeps hunters at session effort.

   Verified by an independent fresh-context check (confirmed, risk low): gh issue view 6909 (no owner comment); origin/main plugins/bugs/skills/scan/SKILL.md:144-176 (:155 hunters pass no effort); git log origin/main plugins/bugs/skills/scan (#7265, #7235, #6913); gh pr list --search 'bugs:scan recall': none building the eval; AGENTS.md Code Review Rules: a cut ships only with eval evidence of no loss

   Admission to an agent lane stays with the work-class rules and owner ratification.
5. kyle-sexton commented on Oct 13, 2026

   

   kyle-sexton

   on Oct 13, 2026

   MemberAuthor

   More actions

   Owner batch-ratified this admission in an attended session on 2026-10-12: work-class: read-only. Rule: class C1-C3, no trust surface, the independent check confirmed the recorded decision, and no owner step remains.
6. kyle-sexton

   added

   work-class: read-onlyAudits, research, reports. No repository mutation; tracker and queue writes only.Audits, research, reports. No repository mutation; tracker and queue writes only.

   agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.

   and removed

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.

   on Oct 13, 2026
7. melodic-automation-lanes

   added

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   and removed

   agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.

   on Oct 14, 2026
8. melodic-automation-lanes commented on Oct 14, 2026

   

   melodic-automation-lanesBot

   on Oct 14, 2026

   Contributor

   More actions

   The work-resolve-issue lane did not start this issue: it needs a person to make it clearer first.

   Why, in the run's own words:

   The issue leaves three decisions open, so I started nothing. (1) Done-ness: is the deliverable only the recall fixture and its scorer, or also the measured comparison and the decision to lower hunter effort? The comparison needs live model runs of bugs:scan at session effort and at 'low', which an unattended CI lane cannot make or verify. (2) The 'low' arm: bugs:scan has no knob that sets hunter effort (SKILL.md says hunters pass no 'effort'). Should the 'low' arm be a temporary SKILL.md variant, an eval-harness arm override, or something else? (3) 'Recall holds' has no threshold: how many seeded bugs across how many fixtures, how many repeat runs per arm given run-to-run variance, and what maximum recall drop (for example no drop, or within N points) allows the cut? Suggested split: one issue for the seeded-bug fixture, scorer and baseline run, and a separate human-run issue for the measurement and the effort change. Please confirm that split or answer the three points.
   

   Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/38627014532

   To retry, answer the question here, remove needs-human and add agent-ready again.
9. kyle-sexton

   added the

   Task

   issue type on Oct 15, 2026
10. kyle-sexton commented on Oct 18, 2026

    

    kyle-sexton

    on Oct 18, 2026

    MemberAuthor

    More actions

    This was generated by AI during backlog shaping.

    Closing as not planned. The owner's recorded decision keeps bugs:scan hunters at session effort and says to close until cost data shows hunters dominate spend.

    Evidence:

    • feat(bugs): measure hunt recall before lowering bugs:scan hunter effort #6909 (Decision recorded comment, 2026-10-13)
    • plugins/bugs/skills/scan/SKILL.md:155

    The owner approved this in an attended backlog-shaping session on 2026-10-17.
11. kyle-sexton

    closed this as not plannedon Oct 18, 2026
12. kyle-sexton commented on Oct 18, 2026

    

    kyle-sexton

    on Oct 18, 2026

    MemberAuthor

    More actions

    This was generated by AI during backlog shaping.

    Reopened, and parked rather than closed. This issue is the evidence gate for lowering bugs:scan hunter effort, so closing it as not planned would lose that gate. It stays parked (needs-human) until there is cost data, as the earlier decision said. Done on the owner's go-ahead in an attended backlog-shaping session, 2026-10-18.
13. kyle-sexton

    reopened this on Oct 18, 2026