Skip to content

docs(adr): record that verification guards earn default-on by measured precision #1352

Description

@kyle-sexton

Observation

The #1270 guard program produced a reusable lesson that currently lives only in closed PRs and issue bodies. It should be an ADR.

Three guards were scoped on sound oracles. Two did not survive contact with the repository:

The generalizable finding: a sound oracle is necessary but not sufficient for shipping a guard default-on. The path guard's oracle was exact � no candidate resolved at the repo root. Its scope was wrong: a repo whose docs mostly describe other repos' trees. A passing contract suite proves the oracle; only a corpus sweep proves the scoping.

Direction (no open decision)

Add docs/adr/0003-verification-guards-earn-default-on-by-measured-precision.md recording the decision that a verification guard measures its firing rate and precision against a real corpus before shipping default-on, that the number appears in the PR, that zero true positives disqualifies regardless of oracle soundness, and that "wrong oracle" versus "wrong scope" decides deletion versus rescoping.

It extends ADR 0002's verification-promotion discipline one step earlier in the lifecycle: 0002 governs promoting an advisory gate to blocking on demonstrated precision, this governs whether it ships default-on at all, on the same evidentiary basis. docs/conventions/hook-precision/README.md keeps the post-ship over-fire discipline.

The ADR also records the cost honestly: measuring immediately after the guard first worked, rather than after polish and review, would have saved a full review round.

Acceptance

  • ADR exists, follows the house format of 0001/0002 (# Title, Status, Date, Context, Decision, Consequences)
  • Every cross-reference resolves
  • It defers to hook-precision/README.md rather than restating it
  • The withdrawn guards' evidence is cited, not duplicated

Related

#1270 (scoping), #1284 (closed), #1319 (shipped guard), #1314 (withdrawn guard + measurement).


This was generated by AI during a session on the fact-check capability.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.priority: lowNice-to-have, cosmetic, or speculative; opportunistic.status: readyTriaged, unblocked, and fully specified; eligible to pick up.work-class: read-onlyAudits, research, reports. No repository mutation; tracker and queue writes only.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions