source-control:babysit-prs: one shared authorship/finding/approval classifier + golden fixtures; gate script owns the merge-ready boolean (6-issue class)

[kyle-sexton]

Category: refactor (umbrella) · Source: control-tower gap-hunt 2026-07-19 (signal mining + skill-catalog audit)

Pattern — merge-readiness classification is ad-hoc per call site, and the model re-derives what a script should own

Six open issues are the same defect class — the babysit snapshot/gate misclassifies authorship/finding/approval state:

• source-control:babysit-prs: resolve-thread reports humanThreadsActed=1 for a Bot-authored thread (mislabeled counter) #512 bot thread counted as humanThreadsActed
• source-control:babysit-prs: snapshot classifies an Approve-with-nits claude[bot] review as blocking bot feedback (gate correctly returns findings=0) #499 Approve-with-nits read as blocking
• source-control:babysit-prs: babysit-readiness-gate.sh over-counts lifetime P-badges as 'findings', false READINESS_BLOCKED reason=under-decomposed on fully-classified PRs #465 lifetime P-badges over-counted as findings → false READINESS_BLOCKED
• source-control:babysit-prs: needs_worker's new_human_blocking_feedback fires on the worker's own self-authored replies (same-login false positive) #473 self-authored replies counted as human feedback
• babysit-prs: single---pr snapshot mode leaves self_logins empty (self-filter + foreign-activity dormant) #497 single---pr mode leaves self_logins empty
• source-control:babysit-prs: safety.md's 'never retry a harness permission denial' rule doesn't match auto-mode classifier's actual (retriable) behavior #455 retry-classifier contract mismatch

Root cause: authorship/finding/approval detection is hand-rolled at each call site; no shared classifier; no fixture tests. Consequence documented in the lane prompt itself: because the gate script is unreliable, the prompt instructs the model to cross-check live thread state each cycle — the model re-derives a boolean predicate (green + issue-linked + lane-authored + no human block + no do-not-merge + head-unchanged + author≠approver) that should be script-owned.

Proposed fix

1. Extract ONE shared classifier (authorship, finding, approval state) used by snapshot, readiness gate, and needs_worker.
2. Golden PR-state fixtures: bot-vs-human thread, nit-approval, resolved finding, self-reply, single-PR mode — every member issue above becomes a fixture case; regression-proof by construction.
3. babysit-readiness-gate.sh owns the merge-ready boolean end-to-end; the model consumes its verdict and never re-evaluates. This is a precondition for trusting feat(source-control): babysit autopilot merge tier — bot-account review + criteria-gated merge #476 autopilot merges.

Relations

Members above; #476 (autopilot trusts this boolean); #475/#484 (adjacent gate-script contract bugs); #479 (absorption umbrella — the prompt's cross-check workaround gets deleted when this lands).

[kyle-sexton]

This was generated by AI during triage. · Category: refactor (umbrella) · Priority: priority: high (precondition for trusting #476 autopilot merges; consolidates a 6-issue misclassification class). Routed needs-human + wayfind: design.

Why needs-human

Extracting ONE shared authorship/finding/approval classifier used by snapshot + readiness gate + needs_worker, and making babysit-readiness-gate.sh own the merge-ready boolean end-to-end, is an architecture refactor — a design decision about the classifier's contract and where the boolean authority lives, not a defaultable one-liner.

The design

1. One shared classifier (authorship, finding, approval state) — single source of truth, replacing per-call-site hand-rolled detection.
2. Golden PR-state fixtures — bot-vs-human thread, nit-approval, resolved finding, self-reply, single-PR mode: every member issue becomes a regression fixture by construction.
3. Gate owns the merge-ready boolean end-to-end (green + issue-linked + lane-authored + no human block + no do-not-merge + head-unchanged + author≠approver); the model consumes the verdict and NEVER re-evaluates — deleting the prompt's current "cross-check live thread state each cycle" workaround.

Compose with already-routed member fixes (do NOT double-fix)

The 6-issue class: #512 (bot thread counted as human) and #499 (Approve-with-nits read as blocking) were routed status: ready this session as individual fixes; #465 (P-badge over-count), #473 (self-reply counted), #497 (single---pr self_logins empty), #455 (retry-classifier contract) are the rest. Recommendation: land those fixes AS the first golden fixtures of the shared classifier rather than patching each call site separately — the ready-routed ones become fixture case #1 and #2. Implementer composes; this umbrella is where the shared classifier + fixtures live.

Relations: #476 (autopilot trusts this boolean — hard dependency), #475/#484 (adjacent gate-script contract bugs), #479 (absorption umbrella — the prompt's cross-check workaround gets deleted when this lands). Decompose candidate (classifier extraction → fixtures → gate-owns-boolean).

[kyle-sexton]

Decision brief (2-minute read)

Problem. Authorship / finding / approval detection is hand-rolled independently in three babysit surfaces that must agree but don't, and the six member issues are divergences among them. The decision is where the one shared classifier lives, which script is authoritative for the merge-ready boolean, and how to cross the bash/Python boundary — not simply "give the gate a boolean it lacks."

Today (verified). There are three independent classifiers, not one gate missing a boolean:

1. Snapshot / needs_worker — skills/babysit-prs/scripts/babysit_delta.py (classify_pr, self_logins) + babysit_feedback.py (actor typing, approval/blocking regexes APPROVAL_VERDICT_RE, BLOCKING_TEXT_RE, self-login filtering). This is where source-control:babysit-prs: snapshot classifies an Approve-with-nits claude[bot] review as blocking bot feedback (gate correctly returns findings=0) #499 (Approve-with-nits read as blocking), source-control:babysit-prs: needs_worker's new_human_blocking_feedback fires on the worker's own self-authored replies (same-login false positive) #473 (self-replies counted as human feedback), babysit-prs: single---pr snapshot mode leaves self_logins empty (self-filter + foreign-activity dormant) #497 (single---pr leaves self_logins empty) live.
2. Finding-decomposition pre-gate — scripts/babysit-readiness-gate.sh, a bash grep/jq re-implementation of finding-counting + self-author + classification detection (R1/R5/R6). Pure predicate, no writes. source-control:babysit-prs: babysit-readiness-gate.sh over-counts lifetime P-badges as 'findings', false READINESS_BLOCKED reason=under-decomposed on fully-classified PRs #465 (lifetime P-badges over-counted) lives here.
3. Merge-ready boolean — skills/babysit-prs/scripts/babysit_merge.py evaluate() (via wrapper bin/source-control-babysit-merge) already owns the real predicate: mergeStateStatus==CLEAN + branch rules + review decision + unresolved threads + check rollup + head-pin + author-is-self (self_logins) + unprotected-base / dependency-author holds. Pure predicate by default; --merge is a separate opt-in. See SKILL.md:185-201.

So the issue's phrasing — "babysit-readiness-gate.sh owns the merge-ready boolean" — mis-names the owner. readiness-gate.sh only gates finding-decomposition; the merge boolean already lives in babysit_merge.py. Each of the three surfaces re-derives "who is self," "what is a finding," "what is approval vs blocking" separately — the smoking gun is #499: on the same PR the gate correctly returned findings=0 while the snapshot classifier reported "1 blocking bot finding." Same ground truth, opposite verdicts, because separate code. There are no golden fixtures for any of the three. The "model re-derives the boolean each cycle" narrative is real but implicit: orchestration.md:14 already mandates trusting the deterministic engine ("not something to re-derive by eyeballing"), and the unreliable classifier forces the model to override it anyway (#473 documents the worker "re-deriving this from scratch").

Options.

Option A — one shared Python classifier module; babysit_merge.py stays the boolean owner; bash gate shells out (RECOMMENDED).
Extract authorship (self / bot / human), finding (severity occurrence + lifetime-vs-open state), and approval-verdict detection into one Python module. babysit_delta.py/babysit_feedback.py and babysit_merge.py import it directly; babysit-readiness-gate.sh shells out to a Python entrypoint for finding-counting instead of re-implementing grep rules — precedent already exists (source-control-babysit-merge is a bash wrapper over babysit_merge.py). babysit_merge.py remains the merge-ready boolean owner (it already integrates CLEAN + branch rules + threads + head-pin + author≠self); it just stops carrying its own private authorship copy. Each member issue becomes a golden fixture case by construction (bot-vs-human thread, nit-approval, resolved-vs-lifetime finding, self-reply, single-PR mode). The model consumes verdicts and stops overriding them; the implicit re-derivation goes away.

• Tradeoffs: real work — one new module + migrating three call sites + a fixture corpus; must resolve the bash/Python language boundary explicitly (shell-out vs absorb readiness-gate.sh into Python entirely — recommend shell-out to preserve the existing pure-predicate CLI contract).
• Future-state risk: low and shrinking — a shared classifier is where feat(source-control): babysit autopilot merge tier — bot-account review + criteria-gated merge #476 autopilot's trust must anchor; leaving three copies means autopilot merges on whichever copy is wrong.
• Reversibility: high — internal refactor behind stable script CLIs; fixtures are the safety net.
• Rationale: fixes the root cause (divergent copies) once, and is the precondition feat(source-control): babysit autopilot merge tier — bot-account review + criteria-gated merge #476 explicitly depends on.

Option B — patch the six member issues in place, no shared classifier.
Fix each call site individually; add fixtures only where cheap.

• Tradeoffs: fastest to green per issue, but leaves three copies that will re-diverge; the triage comment already warns against this ("rather than patching each call site separately"). Confidently ships the same class of bug again.
• Reversibility: high per-patch but the structural debt compounds. Not recommended — treats symptoms.

Option C — collapse all three surfaces into a single end-to-end gate script emitting the whole merge-ready boolean.
Fold snapshot classification + finding-decomposition + babysit_merge.py into one script; model consumes one verdict.

• Tradeoffs: maximal "one boolean" purity, but over-scoped — it rewrites the merge helper's privileged-capability isolation (its whole point is being a narrow one-capability helper) and couples snapshot-time classification to merge-time gating, which run at different moments with different inputs.
• Future-state risk: high — a monolith is harder to keep as a pure predicate under source-control: no read/consume-only posture — "safe" babysit tier and pull-request monitor write to GitHub autonomously #440's report-only constraint. Not recommended — over-consolidation.

Recommendation: Option A. Decompose as the issue suggests: classifier extraction → migrate the three call sites → golden fixtures (members as cases) → delete the model's implicit re-derivation. Keep every surface a pure predicate with no writes (all three already are) so #440's report-only default holds. Fold #455 in as a reference only, not a fixture — see Related.

Related.

• feat(source-control): babysit autopilot merge tier — bot-account review + criteria-gated merge #476 (autopilot merge tier) — APPROVED, staged, binding sequence (source-control:babysit-prs: gh-bot.sh bot token generation fails, worker writes fall back to personal identity #450 → safety trio babysit-prs: merge-step no-monitor STOP, bare-wrapper pipe rule, snapshot exit-code taxonomy, cross-PR dependency channel #373/fix(source-control): review-trigger head-staleness hardening (BLOCKED-behind guard, stale-head reactions) #327/fix(source-control): require per-thread pins for autonomous bulk babysit thread resolves #326 → tier). source-control:babysit-prs: one shared authorship/finding/approval classifier + golden fixtures; gate script owns the merge-ready boolean (6-issue class) #534's "one trusted boolean the model never re-evaluates" is the precondition feat(source-control): babysit autopilot merge tier — bot-account review + criteria-gated merge #476's autopilot consumes; this is the rationale, not a footnote.
• source-control:babysit-prs: gh-bot.sh bot token generation fails, worker writes fall back to personal identity #450 (bot→personal identity fallback) — decided (surface as observability finding); hard prerequisite chore: initialize marketplace scaffold #1 for feat(source-control): babysit autopilot merge tier — bot-account review + criteria-gated merge #476. Real author≠approver identity is what babysit_merge.py's self_logins check depends on.
• source-control: no read/consume-only posture — "safe" babysit tier and pull-request monitor write to GitHub autonomously #440 (outward_write_mode: report-only|write, default report-only) — decided. Constrains the shared classifier/gate to stay a pure predicate (no writes); all three surfaces already satisfy this — preserve it.
• guardrails: hooks ship false-positive over-fires with no per-hook fixtures or shared precision convention (6-issue class: diff-scope, quoted-arg, boolean-intent, stdin, path-branch) #547 (guardrails hooks: per-hook golden fixtures) — sibling convention; its own triage comment names source-control:babysit-prs: one shared authorship/finding/approval classifier + golden fixtures; gate script owns the merge-ready boolean (6-issue class) #534 as the same fixtures-as-prevention pattern. Share one fixture-harness convention, don't reinvent.
• source-control:babysit-prs: worker-scope per-PR lease did not prevent a concurrent worker on the same PR (duplicate P1 fixes) #524 (worker lease concurrency near-miss) — context: once autopilot trusts one boolean and merges, lease correctness gates safety; not blocking this refactor.
• source-control:babysit-prs: safety.md's 'never retry a harness permission denial' rule doesn't match auto-mode classifier's actual (retriable) behavior #455 (harness-permission auto-mode retry) — different classifier (retry semantics of the auto-mode permission layer, not authorship/finding/approval) and briefed separately. Fold as a reference sibling, not a golden-fixture case of the shared classifier.
• Members source-control:babysit-prs: resolve-thread reports humanThreadsActed=1 for a Bot-authored thread (mislabeled counter) #512 / source-control:babysit-prs: snapshot classifies an Approve-with-nits claude[bot] review as blocking bot feedback (gate correctly returns findings=0) #499 / source-control:babysit-prs: babysit-readiness-gate.sh over-counts lifetime P-badges as 'findings', false READINESS_BLOCKED reason=under-decomposed on fully-classified PRs #465 / source-control:babysit-prs: needs_worker's new_human_blocking_feedback fires on the worker's own self-authored replies (same-login false positive) #473 / babysit-prs: single---pr snapshot mode leaves self_logins empty (self-filter + foreign-activity dormant) #497 — five clean cases of the authorship/finding/approval class; each becomes fixture case 1..5. source-control:babysit-prs: snapshot classifies an Approve-with-nits claude[bot] review as blocking bot feedback (gate correctly returns findings=0) #499 is the divergence proof; source-control:babysit-prs: needs_worker's new_human_blocking_feedback fires on the worker's own self-authored replies (same-login false positive) #473 is CLOSED (fixture-only).

Note for the operator: the one decision that isn't defaultable is the bash/Python boundary in Option A — shell-out (recommended, preserves readiness-gate.sh's CLI) vs absorbing it into Python. Everything else follows the issue's proposed shape.

[kyle-sexton]

Operator decision (2026-07-19, decision-session batch 3; relayed).

ACCEPTED — the decision brief's RECOMMENDED option verbatim.

Re-intake: removing needs-human, adding priority: needs-triage for routing.