Skip to content

orb(trust): six of seven actuation sites write no decision record — cap, nag and approved-merge closes are invisible to the ledger and excluded from the certified guarantee #9134

Description

@JSONbored

Problem

"Every verdict is published with a decision record" is false. buildDecisionRecord /
persistDecisionRecord appear at exactly one call site — src/queue/processors.ts:3426-3447, inside
runAgentMaintenancePlanAndExecute. There are seven callers of executeAgentMaintenanceActions, and
six of them perform real GitHub mutations with no record and therefore no ledger row:

site what it does
processors.ts:2871 maybeCloseForContributorCapOnOpen auto-close on pull_request.opened; processors.ts:6530 short-circuits the whole pipeline before the record site is reached
processors.ts:13887 maybeThrottleReviewNagPing closeKind: "review_nag" close
processors.ts:14078 maybeThrottleMonitoredMentions closeKind: "review_nag" close
agent-approval-queue.ts:414 the staged auto_with_approval merge/close after human accept
processors.ts:3980 prReadyForReview branch write (rebase)
processors.ts:4423 maybeForceFreshRebase branch write (rebase)

The irony is exact: processors.ts:3387-3390 enumerates "contributor cap, blacklist, copycat,
review-nag, screenshot-table, linked-issue hard rule"
as the policy closes that the record's
policy_close:<kind> reason code exists to name — and two of those six have dedicated close paths that
never reach the record site.

Trigger

A contributor opens a 3rd PR while at a 2-PR cap → auto-closed on opened. Then:

SELECT * FROM decision_records WHERE pull_number = <n>;   -- no rows

no decision_ledger row, and the PR comment renders no decision-record collapsible.

Impact

  1. The omission is biased toward exactly the closes a contributor is most likely to dispute — cap and
    nag closes, where the PR itself may be perfectly good — plus every approval-gated merge.
  2. It silently biases loadCalibrationPairs (src/review/risk-control-wire.ts:88-112), which JOINs
    audit labels to decision_records. Labels on cap/nag closes can never join and are dropped from the
    certified guarantee's label set
    — so the published ≥95% close guarantee is computed over a
    population that structurally excludes a class of closes.
  3. An external evaluator checking the accountability claim finds coverage holes on the paths that matter
    most.

Dedup

Distinct from #9003, which asks for a generic observability invariant ("every decision records a
named reason") on dispatch paths. This is specifically the decision-record/ledger coverage gap and its
knock-on effect on the calibration population. Related to the sibling issue on review-nag breaker
coverage, which shares the same root cause (six actuation sites bypassing the disposition sequence).

Requirements

  • Hoist record-building into executeAgentMaintenanceActions — or a wrapper every call site uses — so
    any executed merge/close emits a record + ledger row by construction, rather than depending on
    which entry point performed it.
  • Recompute the risk-control label population once coverage is complete, and state on the public
    guarantee page whether the previously published figure was computed over a biased sample.
  • Add an invariant test that fails if a new executeAgentMaintenanceActions call site is added without
    the record path.

Test Coverage Requirements

99%+ patch coverage, branch-counted. One regression test per previously-uncovered close path asserting a
record and a ledger row exist.

Links & Resources

maintainer-only — trust-surface coverage and calibration integrity.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions