feat(telemetry): gate false-positive measurement loop (#554)

[JSONbored]

Closes #554.

What

Maintainers won't move a gate from advisory to block without evidence it's precise. This records each gate block and computes a per-gate-type false-positive rate (blocked-then-merged / blocked) — the evidence to promote (or hold back) a gate. MEASUREMENT only, mirroring the existing #543 outcome-learning loop: it never auto-adjusts a gate or score.

How (mirrors #543's ledger + outcome-calibration service)

• gate_outcomes ledger (migration 0041): one latest block row per (repo, PR), upserted at the gate-failure finalize site in processors.ts. Best-effort — a write failure never aborts gate finalization (same pattern as the slop-assessment persist). Privacy: repo + PR number + blocker codes + timestamps only — no actor logins, no trust/reward fields.
• Override = strongest signal: a maintainer override (feat(github-app): gate appeal/override command + configurable settings #538) flags the row overridden (a human explicitly judged the block wrong).
• gate-precision service: a blocked PR that later merged is a false positive. Per gate type → blocked / blockedThenMerged / overridden / falsePositiveRate (null below a 5-block sample so a 1-of-1 isn't reported as noise), plus human-readable signals ("keep it advisory until this drops"). Pure builder + env-bound loader, same split as outcome-calibration.
• Exposure: read-only GET /v1/repos/:owner/:repo/gate-precision, the same requireRepoMaintainer auth + canSessionAccessPath allowlist as /outcome-calibration. No public route. A privacy-safe gate_blocked product-usage event is also emitted.

Acceptance criterion met

A blocked-then-merged PR increments the false-positive count (per gate type and overall); the rate is queryable per gate mode via the maintainer-authenticated endpoint.

Tests

gate-precision.test.ts (12 cases): blocked-then-merged increments the count, per-gate-type attribution (a multi-code block counts toward each), the MIN_SAMPLE null guard, overridden tracking, and a privacy assertion that the report carries no actor/PII fields. Plus repository upsert/override/list coverage.

Verification

migration guard (0001..0041, no dup) · typecheck clean · full suite 1993 passed, 1 skipped (only the pre-existing pngjs visual-agent skip) · openapi drift-clean (spec is hand-curated; the sibling /outcome-calibration route isn't in it either).

Relates #525/#528 (Phase 1), #543 (outcome-learning loop), #538 (gate override = the override signal).

[ghost]

Note

Gittensory Gate skipped

PR closed before full evaluation. No late first comment was created.

Signal	Result	Evidence	Action
Gate result	⚠️ Skipped	#828 is no longer open.	No action.

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

❌ Patch coverage is 97.61905% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.56%. Comparing base (07d7dae) to head (944f4e7).

Files with missing lines	Patch %	Lines
src/api/routes.ts	77.77%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #828   +/-   ##
=======================================
  Coverage   96.55%   96.56%           
=======================================
  Files          97       98    +1     
  Lines       14091    14175   +84     
  Branches     5141     5169   +28     
=======================================
+ Hits        13606    13688   +82     
  Misses        105      105           
- Partials      380      382    +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ghost]

gittensory · advisory review

Reviewed 9 changed file(s) — two independent AI reviewers.

Changed files (9)

• migrations/0041_gate_outcomes.sql
• src/api/routes.ts
• src/db/repositories.ts
• src/db/schema.ts
• src/queue/processors.ts
• src/services/gate-precision.ts
• src/types.ts
• test/integration/api.test.ts
• test/unit/gate-precision.test.ts

Suggested action: ✅ Safe to merge — both reviewers found no blocking issues.

Reviewer A · gpt-oss-120b — recommends ✅ merge
This PR introduces a new telemetry service to measure gate false‑positive rates, adding a new DB table, repository functions, service logic, API route, queue hooks, type definitions, migrations, and comprehensive tests. The implementation respects privacy constraints, maintains existing gate behavior, and appears functionally correct.

Suggestions

• Add a unit test for listGateOutcomes when no filter conditions are provided to confirm the where(undefined) path works as expected.
• Consider documenting the expected format of the windowDays query parameter (e.g., max 365) in the API docs for clarity.
• Verify that the new migration (0041_gate_outcomes.sql) is included in the migration rollout pipeline to avoid missing the table in production.

Worth double-checking

• If the where(undefined) call in listGateOutcomes behaves unexpectedly in the ORM, the query could fail for unfiltered listings.
• The new route must remain maintainer‑only; ensure future route‑access changes also include isRepoGatePrecisionPath.

Reviewer B · nemotron-3-120b-a12b — recommends ✅ merge
This PR adds a new telemetry feature to measure gate false-positive rates (blocked-then-merged PRs) for maintainer decision-making. It introduces a new database table (gate_outcomes), service functions to build precision reports, API endpoints, and integrates gate blocking/overriding events into the queue processors. The change is well-scoped, maintainer-only, avoids forbidden public terms, and includes comprehensive unit/integration tests. The implementation follows existing patterns (similar to outcome-calibration) and maintains the public/private boundary.

Suggestions

• Consider adding a comment in loadGatePrecisionReport clarifying that windowDays=null means 'full history' (no filtering) to match the outcome-calibration behavior.
• In the gate-precision API route, explicitly reject negative windowDays values with a 400 response (currently negatives are treated as undefined/full window due to the windowDaysRaw > 0 check).
• Ensure the migration file includes a comment noting it's safe to run multiple times (CREATE IF NOT EXISTS already handles this).

Worth double-checking

• Verify that the new gateOutcomes table doesn't cause performance issues on high-volume repos (indexes on repo_full_name, pull_number, and updated_at should help).
• Confirm that the product usage event 'gate_blocked' doesn't inadvertently expose sensitive data (it only includes blockerCodes, which is safe).
• Double-check that the overridden flag preservation logic in recordGateBlockOutcome (omitting it from the update set) works correctly with SQLite's boolean handling.