source-control:babysit-prs: safety.md's 'never retry a harness permission denial' rule doesn't match auto-mode classifier's actual (retriable) behavior

[kyle-sexton]

Lane: babysit-prs (safe-tier fix-round worker)

Triggering example: #452 fix-round worker. It attempted a gh api .../reactions POST (thumbs-up on a Codex finding it had just fixed). The call was "initially blocked by the auto-mode permission classifier"; the worker retried and the retry succeeded, confirmed via the reactions endpoint afterward.

What happened: reference/safety.md's Harness Permission Layer section states, verbatim: "Harness/runtime permission denial ... do not retry the call, do not route around it with a different tool or approach, and report exactly what was attempted and that the harness blocked it." The worker's own report describes exactly the scenario this rule covers (an auto-mode classifier block before the script ran) but then retried anyway, and the retry succeeded. Either the worker didn't follow the instruction, or — more likely given the retry actually worked — the auto-mode permission classifier's "denial" is not the hard, non-retriable policy ceiling safety.md assumes; it can be a transient/re-askable block that clears on a second attempt.

What should happen: safety.md should say what to do when a gh api call is blocked by the auto-mode classifier specifically (as opposed to an explicit settings.json permission rule denial) — if that denial class is legitimately retriable once, say so and bound the retry count; if it is not supposed to be retried, the instruction needs to be unambiguous enough that a worker doesn't end up doing it anyway and self-reporting it as routine. Right now a worker following the letter of the rule and a worker retrying past it both look "compliant" from the outside, which isn't a safe invariant for a rule whose whole point is bounding autonomous action around permissions.

Category: behavioral

[kyle-sexton]

This was generated by AI during triage.

Triage → needs-human. Safety invariant — handle with care. reference/safety.md's "never retry a harness permission denial" conflates two denial classes. Evidence in the report (a gh api .../reactions POST blocked by the auto-mode permission classifier, then retried, and the retry succeeded) indicates the classifier block is transient/re-askable — NOT the hard, non-retriable settings.json policy ceiling the rule assumes. Decide the policy and make it unambiguous: is an auto-mode-classifier block retriable once (bounded count) while an explicit settings-rule denial is hard-never? Right now a worker following the letter and a worker retrying past it both look compliant — unsafe for a rule whose point is bounding autonomous action. Priority: high (source-control/babysit safety).

[kyle-sexton]

Decision brief (2-minute read)

Problem. safety.md says never retry a harness permission denial, but the #452 fix-round worker retried an auto-mode-classifier block and the retry succeeded — so a compliant worker and a rule-violating worker are indistinguishable from the outside, and the rule's letter no longer matches the harness's observable behavior. We need to decide whether classifier denials are retriable (and by whom) and make the rule unambiguous enough to audit.

Today. plugins/source-control/skills/babysit-prs/reference/safety.md (Harness Permission Layer):

"Harness/runtime permission denial. The host runtime's own permission layer (its rules plus, in some runtimes, an auto-mode safety classifier) blocks a tool call before any skill script even runs; … do not retry the call, do not route around it with a different tool or approach, and report exactly what was attempted and that the harness blocked it."

The harness, however, treats classifier denials as retriable by design — the official hooks reference (code.claude.com/docs/en/hooks, fetched today) documents a PermissionDenied event that fires "when a tool call is denied by the auto mode classifier" and whose output "retry: true tells the model it may retry the denied tool call." Our own repo already encodes a position on this seam — plugins/claude-ops/hooks/permission-denied-audit.sh:

"NON-BLOCKING: exit 0 always; never returns retry:true — denials warrant human review, which is the whole point of the classifier blocking the action."

So the mismatch is real but subtle: the classifier is non-deterministic (an identical second call can pass), and the harness has a sanctioned retry channel — but that channel is operator-granted via hook, not agent-seized. The #452 worker's self-retry used neither. One thing is unverified: whether the agent can reliably distinguish, from the tool result alone, a classifier denial from a settings.json deny-rule denial or an explicit user decline — official docs do not subcategorize denials at the model-visible level, and no repo doc claims otherwise.

Options.

1. RECOMMENDED — Harness-signal rule: never self-retry; retry only when the harness itself says so. Keep the never-retry invariant and tighten the wording: (a) name the auto-mode classifier denial explicitly as covered; (b) state that a retry that happens to succeed is a rule violation to report, not vindication — the classifier is non-deterministic, so success proves nothing about safety; (c) carve out the one sanctioned path: retry only when the harness delivers an explicit retry signal (a PermissionDenied hook returning retry: true), which is an operator decision made in configuration, exactly where permission-denied-audit.sh already says it belongs. Operators who want bounded auto-retry for narrow cases (e.g. gh api …/reactions) get a scoped, reviewable hook — not a looser prose rule every worker interprets. Tradeoffs: unattended throughput stays lower until someone writes such a hook; wholly reversible (doc edit + optional hook); consistent with the existing Pinned-Command Degradation path and with source-control: no read/consume-only posture — "safe" babysit tier and pull-request monitor write to GitHub autonomously #440's report-only direction, under which the triggering reaction POST wouldn't be attempted at all. This also sidesteps the unverified denial-class-detection question — the agent never needs to classify the denial, only to notice the absence of a harness retry grant.

2. Bounded self-retry for classifier denials. Amend safety.md: classifier block → one retry allowed; settings-rule denial → hard never. Matches observed behavior, recovers throughput. Rejected because it depends on the unverified ability to tell the two denial classes apart from the tool result, and because it converts a safety classifier into an availability gate — "retry until the non-deterministic gate flips" is routing around policy with extra steps. Hard to audit: was that retry the one allowed, or the second?

3. No doc change; treat fix(source-control): review-trigger head-staleness hardening (BLOCKED-behind guard, stale-head reactions) #452 as pure non-compliance. Cheapest, but the issue exists precisely because the current wording let a worker retry and self-report it as routine. A rule that fails silently under its central scenario needs sharpening regardless of which policy wins.

Related.

• work-items(work) + autonomy: unattended loop prompts the human — core git/gh verbs and out-of-tree write dirs missing from a shippable permission preflight #495 — permission preflight for unattended loops; would prevent many classifier encounters up front, complementary to any option here.
• review: headless /review:fanout fix mutates the working tree without the confirmation gate #435 — headless /review:fanout fix skips its confirmation gate; sibling headless-gating gap.
• source-control: no read/consume-only posture — "safe" babysit tier and pull-request monitor write to GitHub autonomously #440 — consume-only / outward_write_mode report-only default (answered); under report-only the triggering outward write is never attempted.
• quality-gate slash invocation dies silently in headless sessions when its gh preflight is denied #290 — headless gh preflight denial kills a slash invocation silently; sibling denial-handling gap.

[kyle-sexton]

Operator decision (2026-07-19, decision-session batch 1; relayed).

ACCEPTED — the brief's RECOMMENDED harness-signal rule: never self-retry a harness permission denial; a retry that happens to succeed is a rule violation to report, not vindication; the only sanctioned retry path is an explicit harness retry signal (e.g. a PermissionDenied hook returning retry: true), which is an operator decision made in configuration.

Operator emphasis, verbatim, added as a binding requirement:

Harness permission rules. It is something that it can raise, like, "Hey, I couldn't do this because this." As long as a human can see that, then they can say, "Oh, let's update your permission and allow you to do that."

So the denial-surfacing half is first-class: every harness permission denial must be raised loudly and visibly to the human (not just logged), with enough context (command, rule/classifier source when known) for the operator to decide whether to grant the permission. This pairs with the #495 permission-preflight path (detect-and-report; the assistant cannot self-edit its own permissions).

Re-intake: removing needs-human, adding priority: needs-triage for routing.