feat(calibration): shared ORB+AMS signal-tracking module, with AMS wired live - #8079
Merged
Merged
Conversation
…red live Closes #7982. New packages/loopover-engine/src/calibration/signal-tracking.ts: a storage-agnostic RuleFiredEvent/HumanOverrideEvent + SignalStore interface, mirroring auto-tune.ts's own FlagStore-injection precedent, plus computeRulePrecision/computeRuleRepeatCount -- the primitives #7983 (same-rule repeat alarm) and #7984 (per-rule precision tracking) build on. No behavior change to either consumer's existing decision logic; this is a new, additive signal path. Two adapters, each wrapping EXISTING storage rather than a new table: - ORB: src/review/signal-tracking-wire.ts over audit_events (new db/repositories.ts helper: listAuditEventsByType). - AMS: packages/loopover-miner/lib/signal-tracking-store.ts over the local event-ledger. AMS is wired live (not deferred): discover-cli.ts's real (non-dry-run) run now records a rule-fired signal for every eligibility exclusion filterCandidatesByProfiles produces, closing the exact gap #7982's own audit found -- that filter was previously a pure function with zero persistence, so there was no way to later ask how often an exclusion was actually right. Best-effort throughout (a write/open failure never aborts discovery); never recorded on --dry-run. packages/loopover-miner/docs/ams-signal-tracking-design.md documents what's live vs. the one known, explicitly out-of-scope gap: AMS has no natural human-override trigger yet (excluding a candidate produces no AMS-authored artifact a human can act on the way reopening/reverting a PR does for ORB) -- flagged as a concrete follow-up, not blocking #7983/#7984/#7986 (they can compute real reports off fired-only data; precision just reads null/undecided until an override path exists). Also fixes a real test-isolation leak discovered while wiring this up: test/unit/miner-discover-cli.test.ts had no default-store redirect for the event ledger (unlike its existing portfolio-queue/policy-cache redirects), so running its eligibility-filtering tests wrote real rows into the developer's own ~/.config/loopover-miner/event-ledger.sqlite3.
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
loopover-ui | f242dd1 | Commit Preview URL Branch Preview URL |
Jul 22 2026, 08:39 PM |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #8079 +/- ##
==========================================
- Coverage 91.98% 91.97% -0.02%
==========================================
Files 748 751 +3
Lines 76617 76709 +92
Branches 23239 23268 +29
==========================================
+ Hits 70474 70550 +76
- Misses 5041 5053 +12
- Partials 1102 1106 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
|
24 tasks
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #7982.
Summary
packages/loopover-engine/src/calibration/signal-tracking.ts: a storage-agnosticRuleFiredEvent/HumanOverrideEvent+SignalStoreinterface, mirroringsrc/review/auto-tune.ts's ownFlagStore-injection precedent, pluscomputeRulePrecision/computeRuleRepeatCount— the primitives Same-rule repeat alarm: fast-path detection when one deterministic rule rejects N distinct targets in a short window #7983 (same-rule repeat alarm) and Per-rule (not just per-project) precision tracking in the gate-eval pipeline #7984 (per-rule precision tracking) will build on directly. No behavior change to either consumer's existing decision logic — purely additive.src/review/signal-tracking-wire.tsoveraudit_events(newdb/repositories.tshelperlistAuditEventsByType, using the existing(event_type, created_at)index), and AMS'spackages/loopover-miner/lib/signal-tracking-store.tsover the localevent-ledger.ts.discover-cli.ts's real (non---dry-run) run now records a rule-fired signal for every eligibility exclusionfilterCandidatesByProfilesproduces. This closes the exact gap the issue's own audit found — that filter was previously a 100% pure function with zero persistence, so there was no way to later ask how often an exclusion was actually right. Best-effort throughout (a store-open or write failure never aborts discovery); never recorded on a dry run (matches the dry-run path's own no-op portfolio-queue precedent).packages/loopover-miner/docs/ams-signal-tracking-design.mddocuments what's live vs. the one known, explicitly out-of-scope gap: AMS has no natural human-override trigger yet (excluding a candidate produces no AMS-authored artifact a human can act on, unlike reopening/reverting a PR for ORB) — flagged as a concrete follow-up issue, not a blocker for Same-rule repeat alarm: fast-path detection when one deterministic rule rejects N distinct targets in a short window #7983/Per-rule (not just per-project) precision tracking in the gate-eval pipeline #7984/Replace the blanket CONCRETE_EVIDENCE_BLOCKER_CODES breaker exemption with a per-rule track record #7986 (they can compute real reports off fired-only data; precision just readsnull/undecided until an override path exists —computeRulePrecision's own contract already represents that correctly).test/unit/miner-discover-cli.test.tshad no default-store redirect for the event ledger (unlike its existing portfolio-queue/policy-cache redirects), so its eligibility-filtering tests were writing real rows into the developer's own~/.config/loopover-miner/event-ledger.sqlite3on every run. Fixed with the sameLOOPOVER_MINER_CONFIG_DIRredirect +closeDefaultEventLedger()reset patterntest/unit/miner-event-ledger.test.tsalready established.Test plan
npm run typecheck,npm run build --workspace @loopover/engine,npm run build:mcp,npm run build:minerpackages/loopover-engine's ownnpm test(node:test againstdist/): 606/606 passing, including 9 newsignal-tracking.test.tscasestest/unit/signal-tracking-wire.test.ts(ORB adapter, 16 cases, 100% line/branch on the new file),test/unit/miner-signal-tracking-store.test.ts(AMS adapter, 12 cases),test/unit/miner-discover-cli.test.tsadditions (5 new cases covering the wiring: records on exclusion, no-op when nothing excluded, never records on dry-run, degrades on store-open failure, per-event write failures don't stop the rest)ls ~/.config/loopover-miner/event-ledger.sqlite3confirmed absent after every full-suite run post-fixnpm run test:coverage: 1091/1091 files, 20362 tests, 0 failuresnpm run engine-parity:drift-check: clean, no version bump needed (no gate-decision twin-pair file touched)