Skip to content

orb(dashboard): exclude dry-run shadow actions from the gate-outcome breakdown #9694

Description

@JSONbored

⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.

Context

src/services/gate-outcome-breakdown.ts:2-3 states the contract: "repo-scoped 'agent.action.{merge,close,hold}' audit rows — auto-merged, auto-closed, and held/manual terminal dispositions only."

classifyGateOutcomeAuditBucket (:37-43) accepts a row when TERMINAL_AUTO_OUTCOMES = new Set(["success", "completed"]) contains its outcome. A dry-run shadow lands in exactly that set: src/services/agent-action-executor.ts:423 does const auditOutcome = outcome === "dry_run" ? "completed" : outcome; and src/settings/agent-execution.ts:92 emits the shadow under eventType: agent.action.${input.actionClass}, carrying the discriminator only in metadata.mode = "dry_run". The feeding query (src/db/repositories.ts:3757-3771) filters on eventType + createdAt + repo prefix and groups by (eventType, outcome) only — it never selects or filters metadata_json. classifyGateOutcomeAuditBucket receives just { eventType, outcome }, so it is structurally incapable of excluding them.

Result: a repo running with agentDryRun on that plans 12 merges performs zero GitHub mutations, yet the maintainer quality dashboard reports autoMerged: 12, rates.autoMerged: 100, and the summary string "12 gate outcome(s) ... 12 auto-merged". A maintainer evaluating whether to leave dry-run mode is shown numbers that describe actions that never happened.

The sibling reading the same table gets this right — src/services/public-accuracy-trend.ts:125: AND COALESCE(json_extract(metadata_json, '$.mode'), 'live') <> 'dry_run'.

Requirements

  • The rollup query at src/db/repositories.ts:3757-3771 adds the same COALESCE(json_extract(metadata_json, '$.mode'), 'live') <> 'dry_run' predicate src/services/public-accuracy-trend.ts:125 uses — same spelling, same COALESCE default of 'live' so a legacy row without a mode key is still counted.
  • Because the current query is built with the Drizzle query builder, the predicate must be added as a sql fragment inside the existing and(...), not by rewriting the query into a raw string.
  • classifyGateOutcomeAuditBucket and buildGateOutcomeBreakdown keep their current pure { eventType, outcome, count } input shape — the exclusion belongs in the query, exactly as it does for the accuracy trend. Do not add a mode field to GateOutcomeAuditRollup.
  • A repo with only dry-run actions in the window must produce total: 0 and the existing "No gate-outcome audit events in the last N day(s) for the scoped repos." summary (gate-outcome-breakdown.ts:70), not a zeroed-but-nonzero-total report.

⚠️ Required pattern: src/services/public-accuracy-trend.ts:118-136's loadReversalDayRows is the precedent — the identical COALESCE(json_extract(metadata_json, '$.mode'), 'live') <> 'dry_run' clause on the same agent.action.close/agent.action.merge population. It does NOT satisfy this issue to filter in TypeScript after loading the rows (the query groups by (eventType, outcome) and never returns metadata_json, so there is nothing to filter on); to add a mode column to the rollup type and branch in the pure builder; or to change what agent-action-executor.ts:423 writes — the completed rewrite is deliberate and other consumers depend on it.

Deliverables

  • The gate-outcome rollup query in src/db/repositories.ts carries the dry-run exclusion, and the string '$.mode' appears in it (grep-verifiable).
  • A named regression test seeds one agent.action.merge row with metadata_json {"mode":"dry_run"} and one with {"mode":"live"} for the same repo, and asserts the rollup returns count: 1 for that event type.
  • A row whose metadata_json has no mode key at all is still counted (the COALESCE default arm) — asserted by a third seeded row in the same test.
  • buildGateOutcomeBreakdown over a dry-run-only window returns total: 0 and the no-events summary string — asserted by an end-to-end case exercising the loader, not only the pure builder.

All Deliverables above are required in a single PR. A PR that satisfies only some of them — for example adding the SQL predicate without the no-mode-key coverage arm — does not resolve this issue.

Test Coverage Requirements

99%+ Codecov patch coverage, branch-counted, on src/**. src/db/repositories.ts and src/services/gate-outcome-breakdown.ts are inside coverage.include. The three mode states (dry_run, live, absent) must each be exercised, since the COALESCE default is a real branch of the predicate.

Expected Outcome

The maintainer quality dashboard's auto-merged / auto-closed / held tiles count only actions that actually mutated GitHub. A repo in dry-run mode reports zero gate outcomes instead of a full-looking breakdown of things that never happened.

Links & Resources

src/services/gate-outcome-breakdown.ts:1-7, :37-43, :68-72; src/db/repositories.ts:3757-3771; src/services/agent-action-executor.ts:423; src/settings/agent-execution.ts:88-92; src/services/public-accuracy-trend.ts:118-136.

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions