Skip to content

fix(metrics): exclude dry-run shadow actions from the gate-outcome breakdown - #9791

Merged
JSONbored merged 1 commit into
JSONbored:mainfrom
shin-core:fix/gate-breakdown-dryrun-9694
Jul 29, 2026
Merged

fix(metrics): exclude dry-run shadow actions from the gate-outcome breakdown#9791
JSONbored merged 1 commit into
JSONbored:mainfrom
shin-core:fix/gate-breakdown-dryrun-9694

Conversation

@shin-core

Copy link
Copy Markdown
Contributor

What & why

The gate-outcome breakdown's contract is "auto-merged, auto-closed, and held/manual terminal dispositions only". But listGateOutcomeAuditEventRollups (src/db/repositories.ts) counted every agent.action.{merge,close,hold} row whose outcome is terminal — and a dry-run shadow lands in exactly that set: agent-action-executor.ts rewrites a dry-run's outcome to "completed", and the only discriminator is metadata.mode = "dry_run". The rollup query groups by (eventType, outcome) and never reads metadata_json, so shadow actions inflated the auto-merge/close/hold counts, and a repo whose window held only dry-run actions produced a zeroed-but-nonzero-total report instead of the "no events" summary.

The fix

Add the same dry-run exclusion the public accuracy trend already applies (public-accuracy-trend.ts's loadReversalDayRows) as a sql fragment inside the existing and(...) — not a raw-string rewrite:

sql`coalesce(json_extract(${auditEvents.metadataJson}, '$.mode'), 'live') <> 'dry_run'`

Same COALESCE default of 'live', so a legacy row without a mode key is still counted. classifyGateOutcomeAuditBucket and buildGateOutcomeBreakdown keep their pure { eventType, outcome, count } shape — the exclusion belongs in the query, exactly as it does for the accuracy trend. No mode field added to the rollup type; no change to what agent-action-executor.ts writes.

Tests (test/unit/gate-outcome-audit-rollups.test.ts)

  • A dry-run shadow merge is excluded while a live merge and a legacy (no-mode) close are counted — the merge count is 1, not 2 (fails on main).
  • A repo whose window holds only dry-run actions returns [] (fails on main) — which, via buildGateOutcomeBreakdown([]) (already covered), yields total: 0 and the "No gate-outcome audit events…" summary rather than a zeroed report.

Validation

  • npm run typecheck green; the rollup suite green.
  • Diff coverage on the changed query is complete (the fragment executes in both new tests); no uncovered added lines/branches.
  • git diff --check <base> HEAD clean; diff is two files, no route/schema/migration change (a pure query predicate).

Closes #9694

…eakdown

`listGateOutcomeAuditEventRollups` counted every agent.action.{merge,close,hold}
row whose outcome is a terminal disposition, but a dry-run shadow lands in exactly
that set: agent-action-executor rewrites its outcome to "completed" and the mode
discriminator lives only in metadata.mode. So a repo's gate-outcome breakdown
inflated its auto-merge/close/hold counts with shadow actions that never touched
a PR — and a repo whose window held only dry-run actions produced a zeroed-rate
report instead of the "no events" summary.

Add the same dry-run exclusion the public accuracy trend already applies
(loadReversalDayRows) as a `sql` fragment inside the existing `and(...)`:
`coalesce(json_extract(metadata_json, '$.mode'), 'live') <> 'dry_run'` — same
COALESCE default of 'live' so a legacy row without a mode key is still counted.
The pure builders and the rollup type are unchanged; the exclusion is in the query.

Closes JSONbored#9694
@shin-core
shin-core requested a review from JSONbored as a code owner July 29, 2026 08:47
@superagent-security superagent-security Bot added the contributor:flagged Contributor flagged for review by trust analysis. label Jul 29, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

🚨 Contributor flagged. Click here for more info: Superagent Dashboard

@loopover-orb

loopover-orb Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Warning

⏸️ LoopOver review result - manual review recommended

Review updated: 2026-07-29 09:05:36 UTC

2 files · 1 AI reviewer · no blockers · CI green · unstable

⏸️ Suggested Action - Manual Review

Review summary
This adds a `metadata_json` mode discriminator to the existing `and(...)` filter in `listGateOutcomeAuditEventRollups`, excluding dry-run shadow actions (rewritten to outcome `completed` by agent-action-executor) from the terminal gate-outcome rollup, mirroring the pattern already used in `public-accuracy-trend.ts`. The fix is applied at the correct layer (the query itself) rather than patching the output shape, and the tests exercise the real path: a mixed live/dry-run/legacy-no-mode batch asserting counts, and an all-dry-run-window case asserting an empty array. This is a narrow, well-targeted fix with a plausible before/after bug scenario and solid regression coverage.

Nits — 4 non-blocking
  • The PR body doesn't explicitly state a linked issue number in the diff/description shown, though the test file references 'orb(dashboard): exclude dry-run shadow actions from the gate-outcome breakdown #9694' — confirm this closes an actual open issue per repo convention.
  • No test for a mixed batch where the ONLY dry-run row is on a different eventType (e.g. hold) to confirm the exclusion generalizes across all three event types, not just merge/close.
  • Consider a short comment or test note cross-referencing `public-accuracy-trend.ts`'s `loadReversalDayRows` so a future reader can see both call sites are meant to stay in lockstep if the mode-default convention ever changes.
  • If `metadata_json` can be malformed/non-JSON for very old rows, verify `json_extract` degrades to NULL (and thus COALESCE to 'live') rather than throwing — likely already true for SQLite but worth a one-line confirmation in the PR description.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #9694
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ❌ 5/25 Preflight is holding this PR: the review lane is unavailable, so it is not ready for automated review.
Contributor workload ✅ 10/10 Author activity: 50 registered-repo PR(s), 35 merged, 0 issue(s).
Contributor context ✅ Confirmed Gittensor contributor shin-core; Gittensor profile; 50 PR(s), 0 issue(s).
Improvement ✅ Minor risk: clean · value: minor
Linked issue satisfaction

Partially addressed
The SQL predicate is correctly added as a sql fragment inside and(...) with the exact COALESCE/mode spelling, and the tests cover dry_run exclusion, live inclusion, the no-mode COALESCE default, and a dry-run-only window returning [] at the repository layer. However, the issue explicitly requires an end-to-end assertion that buildGateOutcomeBreakdown over a dry-run-only window returns total: 0 and

Review context
  • Author: shin-core
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is registered but has no active allocation in the current snapshot.
  • Public profile languages: not available
  • Official Gittensor activity: 50 PR(s), 0 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Await review-lane availability.
  • Then work through the remaining 2 steps in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

Decision record
  • action: hold · clause: success
  • config: 44c9edd884eed5af160369b3400a322f01caebc500ae0a6c7f16221c9c54b5f3 · pack: oss-anti-slop · ci: passed
  • record: e8f2ce4b9b0c2cc786d1e8c3ab97c7f3c5b333012f1612f9ea0b436acc7bdd6e (schema v5, head 8f833f1)

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 29, 2026
@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.24%. Comparing base (0fa52ed) to head (8f833f1).

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #9791       +/-   ##
===========================================
- Coverage   90.30%   77.24%   -13.07%     
===========================================
  Files         913      283      -630     
  Lines      113595    61437    -52158     
  Branches    26963     9134    -17829     
===========================================
- Hits       102584    47454    -55130     
- Misses       9682    13648     +3966     
+ Partials     1329      335      -994     
Flag Coverage Δ
backend 96.85% <ø> (+1.28%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/db/repositories.ts 96.85% <ø> (ø)

... and 764 files with indirect coverage changes

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 29, 2026
@loopover-orb

loopover-orb Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Held for manual review: the gate and required CI are green, but GitHub reports this pull request's mergeable state as unstable because a non-required check or status is not passing, so LoopOver will not auto-merge. A maintainer can resolve the failing check or review and merge manually. This is an automated maintenance action.

@JSONbored
JSONbored merged commit 7df6148 into JSONbored:main Jul 29, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor:flagged Contributor flagged for review by trust analysis. gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

orb(dashboard): exclude dry-run shadow actions from the gate-outcome breakdown

2 participants