fix(review): exclude maintainer PRs from the slop-discrimination alert - #5863
Conversation
The ops-anomaly detector flagged "slop score NOT discriminating" on two repos, reading it as a scoring-calibration problem. It wasn't: the slop scorer's weights, band thresholds, and gate-blocker direction are all internally consistent, and the detector's own unit tests pin the intended "higher band merges less" semantics correctly. The real bug is in buildSlopOutcomeCalibration's population: it pools every resolved PR regardless of author. Maintainer-authored PRs merge by human judgment regardless of severity (the same population settings-preview.ts's includeMaintainerAuthors already excludes from the public surface by default) -- so a repo where the maintainer is heavily active can have its contributor-only, correctly-discriminating signal swamped by a maintainer-PR pool that merges at a high rate independent of score, inverting the blended comparison without the score itself being wrong. Add an additive excludeMaintainerAuthors option (default off, every existing caller stays byte-identical) mirroring the miner/human cohorts gate-precision.ts already applies to the sibling #554 false-positive measurement, and opt the ops-anomaly detector's two call sites into it so the alert -- and the internal ops-stats row shown alongside it -- both reflect the contributor-only signal instead of the confounded one.
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
|
Warning 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 ⏸️ LoopOver review result - manual review recommendedReview updated: 2026-07-14 20:31:49 UTC
⏸️ Suggested Action - Manual Review Review summary Nits — 4 non-blocking
Concerns raised — review before merging
📋 Copy for AI agents — paste into your coding agent
Review context
Contributor next steps
Signal definitions
[BETA] Chat with GittensoryAsk Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands 🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.
|
Summary
buildSlopOutcomeCalibration's population: it pools every resolved PR regardless of author. Maintainer-authored PRs merge by human judgment regardless of severity (the same populationsettings-preview.ts'sincludeMaintainerAuthorsalready excludes from the public surface by default) — so a repo where the maintainer is heavily active can have its contributor-only, correctly-discriminating signal swamped by a maintainer-PR pool that merges at a high rate independent of score, inverting the blended comparison without the score itself being wrong.excludeMaintainerAuthorsoption (default off, every existing caller stays byte-identical) mirroring the miner/humancohortssplitgate-precision.tsalready applies to the sibling feat(github-app): gate false-positive telemetry #554 false-positive measurement, and opted the ops-anomaly detector's two call sites into it so the alert — and the internal ops-stats row shown alongside it — both reflect the contributor-only signal instead of the confounded one.Scope
type(scope): short summaryConventional Commit format.CONTRIBUTING.mdand does not reintroduce GitHub Pages, VitePress,site/, orCNAME.Validation
git diff --checknpm run actionlintnpm run typechecknpm run test:coveragelocally — 854 test files passed, 0 failed (full unsharded run). New tests cover: the maintainer-PR confound reproducing the false positive (default behavior, byte-identical),excludeMaintainerAuthors:trueremoving the confound and revealing the true signal, a genuine contributor-only non-discriminating score still correctly flagging (not a blanket suppression), the OWNER/MEMBER/COLLABORATOR vs FIRST_TIME_CONTRIBUTOR/unset classification boundary, the option threading end-to-end throughbuildRepoOutcomeCalibration, and bothrunOpsAlerts/computeOpsStatsno longer alerting on the confound while a real contributor-only inversion still does.npm run test:engine-parity— clean, unaffected (this change doesn't touchpackages/loopover-engine).npm audit --audit-level=moderate— clean, 0 vulnerabilities.npm run test:workers/npm run build:mcp/npm run test:mcp-pack/npm run ui:*— not run, see note below.If any required check was skipped, explain why:
src/services/outcome-calibration.ts,src/review/ops-wire.ts, and their unit tests — no Workers runtime, MCP package, API route, OpenAPI schema, or UI code is touched.Safety
UI Evidence
N/A — no UI/frontend/docs change.
Notes