Skip to content

feat(services): miner-vs-human cohort split in the maintainer recap digest - #4625

Merged
JSONbored merged 1 commit into
mainfrom
feat/maintainer-recap-cohort-split-4521
Jul 10, 2026
Merged

feat(services): miner-vs-human cohort split in the maintainer recap digest#4625
JSONbored merged 1 commit into
mainfrom
feat/maintainer-recap-cohort-split-4521

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

⚠️ Stacked on #4623 (#4520) — do not merge before it

This branch is built on top of feat/gate-precision-cohort-split-4520 (PR #4623), because
GatePrecisionReport.cohorts — the upstream field this entire PR consumes — doesn't exist without it. The
diff below will look larger than it is until #4623 merges and this rebases cleanly onto main; at that point
the diff will shrink to just the 6 files this PR actually owns.

Summary

Issue #4521 explicitly asks for this once the upstream gate-eval split lands — it did, in #4520/#4623.

  • MaintainerRecapRepo (per-repo) and RecapReport.totals (aggregate) both gain an optional cohorts field
    — additive, present only when the underlying GatePrecisionReport carried one.
  • buildMaintainerRecap sums the cohort split only across repos that actually had one — a window mixing
    cohort-aware and legacy call sites degrades gracefully (partial adoption never half-reports zeros for the
    repos that didn't opt in).
  • runMaintainerRecapJob now passes includeCohorts: true to loadGatePrecisionReport unconditionally — a
    periodic (default weekly) digest is exactly the "occasional aggregate view, one extra API call is fine"
    case includeCohorts was designed for, unlike a hot webhook path.
  • formatMaintainerRecap gains an additive ## Cohorts section (between Totals and Per-repo), rendered only
    when totals.cohorts is present — e.g. Miner-originated: 1/3 gate false positives (33%).
  • No new identity read anywhere in this PR — it's entirely downstream of feat(ui): add a miner-vs-human breakdown to the Wave-3 gate-precision / reversal-rate / findings-by-category dashboard cards #4520's already-computed split.

Scope note, matching #4520's own: OutcomeCalibration (the calibration half of MaintainerRecapRepoInput)
does not have its own cohort split yet, so reviewed/merged/closed/reversals stay blended-only. Only
the gate-precision half of the recap gains the split, since that's the half with the upstream field.

Fixes #4521

Test plan

  • test/unit/maintainer-recap.test.ts: 4 new cases — cohorts absent (byte-identical), per-repo split
    attached verbatim + aggregated into totals, summing correctly across multiple cohort-aware repos
    (including a real "n/a" zero-blocked cohort), and graceful degradation when only SOME repos in the
    window carried a split
  • test/unit/maintainer-recap-format.test.ts: 2 new cases — the ## Cohorts section is absent by
    default and correctly rendered (with placement between Totals/Per-repo) when present
  • test/unit/maintainer-recap-wire.test.ts: 1 new end-to-end case seeding a real gate-blocked PR
    with a confirmed-miner author and a second with a human author, proving the split reaches the finished
    report AND the formatted digest through the actual production wiring (runMaintainerRecapJob) — not
    just that the plumbing doesn't crash. All 25 pre-existing tests in this file still pass unchanged
    (they now implicitly exercise includeCohorts: true too, since it's unconditional, but degrade to a
    zeroed cohorts object since none of them seed a gate block)
  • npx tsc --noEmit clean
  • Full relevant suite: 71/71 across all 7 maintainer-recap* test files

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.11%. Comparing base (da9dba5) to head (8bfd562).
⚠️ Report is 6 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4625      +/-   ##
==========================================
+ Coverage   94.09%   94.11%   +0.02%     
==========================================
  Files         430      432       +2     
  Lines       38222    38370     +148     
  Branches    13931    13989      +58     
==========================================
+ Hits        35965    36113     +148     
  Misses       1600     1600              
  Partials      657      657              
Files with missing lines Coverage Δ
src/review/maintainer-recap-wire.ts 100.00% <ø> (ø)
src/services/maintainer-recap.ts 100.00% <100.00%> (ø)
src/types.ts 100.00% <ø> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 10, 2026
@loopover-orb

loopover-orb Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review result - fixes required

Review updated: 2026-07-10 08:16:48 UTC

12 files · 1 AI reviewer · no blockers · readiness 100/100 · CI failing · blocked

🛑 Suggested Action - Manual Review

  • Touches a guarded path — held for manual review: This PR changes guardrail-protected path(s): src/services/gate-precision.ts (matched src/services/**), src/services/maintainer-recap.ts (matched src/services/**).

Review summary
This PR adds an additive miner-vs-human cohort split to the maintainer recap digest, building on gate-precision.ts's extracted foldGateOutcomes helper to compute miner/human buckets from a fetched-once minerLogins set, then threads the split through maintainer-recap.ts's aggregation, formatMaintainerRecap's new '## Cohorts' section, and the /gate-precision route's opt-in includeCohorts query param. The refactor of buildGatePrecisionReport into foldGateOutcomes is a faithful extraction (same accumulation logic, same MIN_SAMPLE floor applied independently per cohort), and the classification (isMinerAuthoredOutcome) fails safe to 'human' on any unresolvable author, matching the codebase's stated convention. Test coverage is thorough (both cohort-present and cohort-absent/legacy arms, case-insensitivity, API-failure degradation, end-to-end wiring), and the change is genuinely additive — every existing call site that doesn't opt in stays byte-identical, which the tests explicitly pin. The one real caveat, openly disclosed by the author, is that this PR is stacked on unmerged #4623 and cannot land standalone since GatePrecisionReport.cohorts doesn't exist on main yet.

Nits — 7 non-blocking
  • This branch is stacked on an unmerged PR (feat(services): miner-vs-human cohort split for the gate-precision dashboard card #4623) and literally cannot compile/merge into main until that lands — worth a merge-order safeguard (e.g. a CI check or draft status) beyond the PR description callout, since description text is easy to miss at merge time.
  • gate-precision.ts:128 and similar comments embed the PR number (feat(ui): add a miner-vs-human breakdown to the Wave-3 gate-precision / reversal-rate / findings-by-category dashboard cards #4520) as an inline marker rather than a tracked constant/changelog entry — harmless but adds noise if this pattern repeats PR-to-PR.
  • maintainer-recap.ts's cohort rate rounding (`Math.round(x*100)/100`) duplicates the existing totals.gateFalsePositiveRate rounding pattern inline rather than factoring a shared `roundRate` helper — pre-existing duplication, not introduced fresh, but the new cohort code was a natural point to consolidate it.
  • routes.ts's `includeCohorts` query flag only accepts the literal string 'true' (like `windowDays`'s existing pattern) — consistent with the file's conventions, just worth confirming that's intentional strictness for API consumers.
  • Once feat(services): miner-vs-human cohort split for the gate-precision dashboard card #4623 lands and this rebases, re-diff to confirm the actual 6-file surface area described in the PR body matches what's reviewed here.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.

CI checks failing

  • validate
  • validate-code
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #4521
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 48 registered-repo PR(s), 40 merged, 334 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 48 PR(s), 334 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Linked issue satisfaction

Partially addressed
The PR extends GatePrecisionReport, MaintainerRecapRepo, and RecapReport.totals with an optional miner-vs-human cohorts field, renders a new '## Cohorts' section that degrades gracefully when absent, and adds cohort-aware/cohort-absent tests plus an explicit no-actor-login privacy test, closely matching most of the issue's deliverables. However, the issue's explicit 'Docs: note the new field in th

Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 48 PR(s), 334 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 10, 2026
…igest (#4521)

Extends MaintainerRecapRepoInput/RecapReport with an additive, opt-in
miner-vs-human split, sourced entirely from the upstream GatePrecisionReport
cohorts field (#4520) -- no independent identity read, no new privacy
surface. A repo's recap entry gets a `cohorts` field whenever its injected
GatePrecisionReport carried one; the aggregate totals.cohorts sums only
across repos that did, so a window mixing cohort-aware and legacy call
sites still degrades gracefully instead of half-reporting zeros.

runMaintainerRecapJob now opts loadGatePrecisionReport into includeCohorts
by default -- a periodic digest is exactly the "occasional aggregate view"
that option was designed for, unlike a hot webhook path.

formatMaintainerRecap gains an additive "## Cohorts" section, rendered only
when totals.cohorts is present, sitting between Totals and Per-repo.

Stacks on #4520 (branch feat/gate-precision-cohort-split-4520, PR #4623) --
GatePrecisionReport.cohorts doesn't exist without it. Do not merge before
#4623.

Fixes #4521
@JSONbored
JSONbored force-pushed the feat/maintainer-recap-cohort-split-4521 branch from f0ac077 to 8bfd562 Compare July 10, 2026 08:21
@JSONbored
JSONbored merged commit 2234cb7 into main Jul 10, 2026
11 checks passed
@JSONbored
JSONbored deleted the feat/maintainer-recap-cohort-split-4521 branch July 10, 2026 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

feat(review): extend MaintainerRecapRepoInput with a miner-vs-human cohort split once gate-eval supports one

1 participant