autonomy: 16 security-binding fixtures (~1,200 lines) are orphaned — no eval references them, no test grades them

[kyle-sexton]

2026-07-20 merged-PR quality audit finding (PR #377 deep-dive; the one DEBT-ACCRUING verdict in an otherwise SOLID day). The 16 fixtures under the autonomy setup skill's evals/fixtures/security-binding/ (equal-specificity-tie, evidence-stale, c2-auto-unratified, missing-l2-autonomous, ...) are referenced by no eval files[] entry, graded by no test, carry no expected-verdict block, and ci.yml names them only to EXCLUDE them from hygiene scans. They look like a golden suite but grade nothing — contrast #634's fixtures, each asserted by a unittest.

Fix directions: (a) attach expected-verdict blocks + wire them into the eval grader per the #634 idiom, or (b) demote them explicitly to documentation exemplars (move out of evals/fixtures/, say so in the doc). Sibling process fix tracked separately: CI orphaned-fixture check.

Note for router: autonomy plugin is WP-lane-owned — coordinate with the WP session rather than dispatching a pipeline work lane blind.

---

Reference: Boris Cherny, "Steps of AI Adoption" (Google Doc)

[kyle-sexton]

This was generated by AI during triage.

Triage (cluster-aware routing) — shares the underlying decision with #664 (standardize the golden-fixture idiom on the graded/#634 shape vs the ungraded/#377 exemplar-pile shape): these 16 orphaned security-binding fixtures are a concrete instance of the not-yet-ratified convention question. Routing #664 as the decision carrier rather than sending this to needs-human individually, per cluster-aware routing.

Reporter's own routing note preserved for whoever picks this up post-ratification: "autonomy plugin is WP-lane-owned — coordinate with the WP session rather than dispatching a pipeline work lane blind."

[kyle-sexton]

Scope note from the #681 orphaned-fixture gate: 7 more ungraded security-binding fixtures landed on main (WP runner-escalation lane) and now fall in this issue's burn-down class — no eval files[] entry and no test grades any of them. They were added to scripts/orphaned-fixtures-baseline.txt when main was merged into the gate branch (commit 57e8c69), tracked here rather than red-lining the gate.

New fixtures now owned by #662:

• plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-ack-invalid.json
• plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-ack-loop.json
• plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-push-without-channel.json
• plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-runner-route-missing.json
• plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-bad-token.json
• plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-tracker-only-valid.json
• plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-valid.json

Cleanup is unchanged in kind: attach expected-verdict blocks and wire them into the grader, or demote them out of evals/fixtures/. Deleting each baseline line as its fixture is graded/removed is what makes the gate's stale-entry check flag the debt as it burns down.