Skip to content

feat(review): add a submission-cadence/inter-arrival-time signal to flag machine-paced submitters - #4549

Merged
JSONbored merged 3 commits into
mainfrom
feat/submission-cadence-signal
Jul 10, 2026
Merged

feat(review): add a submission-cadence/inter-arrival-time signal to flag machine-paced submitters#4549
JSONbored merged 3 commits into
mainfrom
feat/submission-cadence-signal

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Every existing anti-abuse signal (reputation burst-floor, slop score) counts volume or ratios within a time window; none has an inter-arrival-time/rate term. A fast, well-formed, strategically low-value submitter is invisible to the one dimension (superhuman pace) that would otherwise be a strong tell.
  • computeSubmissionCadence / isMachinePacedCadence (pure): the median gap between a submitter's recent review_targets rows, flagged only once both a real sample size (5+) AND a sub-10-minute median gap are present.
  • getSubmitterCadence: queries created_at across ALL review_targets rows, not just terminal ones — a fresh burst of still-open submissions is exactly what this needs to catch, since by the time a PR becomes terminal the (paid) AI review has already run on it.
  • Wired into shouldSkipAiForReputation as an independent, additional check alongside the existing quality/burst signal — a submitter whose individual PRs all look fine can still be caught on cadence alone. The extra cadence read is skipped once the quality/burst check alone already justifies downgrading.
  • New migration adds the supporting (project, submitter, created_at) index — no existing index covers this shape.

Scoped to per-repo cadence for now. An install-wide variant (mirroring #4513's confirmed-miner-aware cross-repo pattern) is a natural fast-follow once #4513 merges — kept out of this PR to avoid depending on an unmerged branch.

Test plan

  • test/unit/submitter-reputation.test.ts — 11 new tests for computeSubmissionCadence/isMachinePacedCadence/getSubmitterCadence (median-gap math, order-independence, even/odd gap counts, boundary thresholds, fail-safe paths)
  • test/unit/reputation-wiring.test.ts — 4 new tests: machine-paced-but-quality-clean submitter is caught, comfortably-human-paced submitter is not, the cadence read is skipped once the quality/burst check already downgrades, null-submitter coalesce
  • 100% statement/branch/function/line coverage on both touched source files
  • npx tsc --noEmit clean

Fixes #4514

@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.08%. Comparing base (80d623f) to head (a9b449a).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4549   +/-   ##
=======================================
  Coverage   94.08%   94.08%           
=======================================
  Files         427      427           
  Lines       37961    37981   +20     
  Branches    13864    13870    +6     
=======================================
+ Hits        35715    35736   +21     
  Misses       1586     1586           
+ Partials      660      659    -1     
Files with missing lines Coverage Δ
src/review/reputation-wire.ts 100.00% <100.00%> (+8.33%) ⬆️
src/review/submitter-reputation.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 10, 2026
@loopover-orb

loopover-orb Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-10 02:42:56 UTC

5 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
AI review could not be completed for this PR head. Gittensory is holding this PR for manual review instead of relying on deterministic signals alone.

Nits — 1 non-blocking
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #4514
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 48 registered-repo PR(s), 40 merged, 347 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 48 PR(s), 347 issue(s).
Gate result ✅ Passing No configured blocker found.
Linked issue satisfaction

Addressed
The PR adds a pure median-inter-arrival-time signal (computeSubmissionCadence/isMachinePacedCadence) plus a D1-backed getSubmitterCadence, wires it as an independent additional check in shouldSkipAiForReputation (advisory-tier, only penalizing sustained sub-10-minute cadence over 5+ submissions, not merged into the existing burst/quality signal), adds a supporting index migration, and includes tes

Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 48 PR(s), 347 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 10, 2026
@JSONbored
JSONbored force-pushed the feat/submission-cadence-signal branch 2 times, most recently from 5aaea42 to c8805ad Compare July 10, 2026 02:10
JSONbored added 3 commits July 9, 2026 19:30
…lag machine-paced submitters

Every existing anti-abuse signal (reputation burst-floor, slop score)
counts volume or ratios within a time window; none has an
inter-arrival-time or rate term. A fast, well-formed, strategically
low-value submitter is, by construction, invisible to the one
dimension (superhuman pace) that would otherwise be a strong tell --
a submitter can clear every quality bar (good outcomes, real
descriptions) while operating at a cadence no human plausibly
sustains, and today that cadence carries zero weight anywhere.

- computeSubmissionCadence / isMachinePacedCadence (pure): the median
  gap between a submitter's recent review_targets rows, flagged only
  once both a real sample size (5+) AND a sub-10-minute median gap
  are present -- a lone fast submission is not a pattern.
- getSubmitterCadence: queries created_at across ALL review_targets
  rows (not just terminal ones -- a fresh burst of still-open
  submissions is exactly what this needs to catch, since the AI
  review already ran on each one by the time it becomes terminal).
- Wired into shouldSkipAiForReputation as an independent, additional
  check alongside the existing quality/burst signal -- a submitter
  whose individual PRs all look fine can still be caught on cadence
  alone, and the (extra) cadence read is skipped once the
  quality/burst check alone already justifies downgrading.

Scoped to per-repo cadence for now, matching this PR's own scope;
an install-wide variant (mirroring #4513's confirmed-miner-aware
cross-repo pattern) is a natural fast-follow once #4513 merges.

Fixes #4514
…4554)

Rebase onto current main and take the next free number now that
migrations/0132_impact_map_query_cache.sql (from #4554, itself a collision
fix) occupies the number this branch originally guessed.
…4556)

Rebase onto current main and take the next free number now that
migrations/0133_screenshot_table_gate_skill_link.sql (from #4556) occupies
the number this branch previously took.
@JSONbored
JSONbored force-pushed the feat/submission-cadence-signal branch from c8805ad to a9b449a Compare July 10, 2026 02:31
@JSONbored
JSONbored merged commit d67d0b5 into main Jul 10, 2026
12 checks passed
@JSONbored
JSONbored deleted the feat/submission-cadence-signal branch July 10, 2026 04:51
JSONbored added a commit that referenced this pull request Jul 10, 2026
…ay collision on main)

main currently has three DIFFERENT already-merged files at 0134
(#4549/#4558/#4563) -- tracked separately as its own fix (PR #4577, not yet
merged). Since that fix already claims through 0138, take 0139 here to
avoid colliding with it once it lands; this branch's own
db:migrations:check will stay red until #4577 merges (unrelated to this
branch's own changes), and will need one more rebase afterward.
JSONbored added a commit that referenced this pull request Jul 10, 2026
…ay collision on main)

main currently has three DIFFERENT already-merged files at 0134
(#4549/#4558/#4563) -- tracked separately as its own fix (PR #4577, not yet
merged). Since that fix already claims through 0138, take 0139 here to
avoid colliding with it once it lands; this branch's own
db:migrations:check will stay red until #4577 merges (unrelated to this
branch's own changes), and will need one more rebase afterward.
JSONbored added a commit that referenced this pull request Jul 10, 2026
…tion ledger (#4577)

* feat(review): persist a login-keyed predict-gate-vs-live-gate calibration ledger

Neither the MCP predict_gate tool nor contributor_gate_history persists
whether a contributor's self-reported predict-gate verdict matched the REAL
gate decision their PR eventually received -- the one calibration signal
the review stack itself uniquely owns, since it terminates both the
self-review call and the live gate for the same login/repo.

predicted_gate_calibration_ledger pairs the most recent predict_gate_calls
row (#4516) for a (login, project) against the real decision at the same
call sites as recordContributorGateDecision, writing one immutable row per
(login, project, pr, commit) -- ON CONFLICT DO NOTHING, never DO UPDATE, so
a webhook replay can never overwrite an already-recorded pairing. Write-only
and server-side only: nothing reads it yet (mirrors contributor_gate_history's
own precedent), and no MCP tool or other contributor-reachable surface can
write to or read from it, preserving its value as anti-farming-resistant
ground truth for a future #2349 consumer.

Depends on #4516 (predicted_gate_calls) -- built stacked on that branch per
the issue's own explicit note that the two may be built together; will be
rebased onto main once that PR merges.

Fixes #4517

* fix(db): renumber ledger migration 0135 -> 0138 (0134/0135 collide on main)

origin/main independently landed three files at 0134 and two at 0135 (from
already-merged PRs #4549, #4558, #4563) since this branch's last rebase.
Rather than touch already-merged migration files here, take the next
genuinely free number for this branch's own new migration; the 0134/0135
collision among already-merged files is a separate main-red issue handled
in its own PR.

* fix(db): resolve migration collision at 0134 on main (already-merged PRs)

Three already-merged PRs independently claimed 0134 (#4549's
review_targets_cadence_idx, #4558's predicted_gate_calls, and #4563's
pr_last_backlog_convergence_regated_at). CI on this branch inherits main's
full migrations/ directory regardless of which PR fixes it, so this can't
be deferred to a separate PR without also blocking this one. Renumber the
two newer files (keeping #4563's oldest 0134 file in place); this branch's
own new migration was already moved to 0138 to stay clear of both this
collision and #4563's separate 0135 collision.
JSONbored added a commit that referenced this pull request Jul 10, 2026
…PRs) (#4581)

Three already-merged PRs independently claimed 0134 (#4549's
review_targets_cadence_idx, #4558's predicted_gate_calls, and #4563's
pr_last_backlog_convergence_regated_at). CI on this branch inherits main's
full migrations/ directory regardless of which PR fixes it, so this can't
be deferred to a separate PR without also blocking this one. Renumber the
two newer files (keeping #4563's oldest 0134 file in place); this branch's
own new migration was already moved to 0138 to stay clear of both this
collision and #4563's separate 0135 collision.
JSONbored added a commit that referenced this pull request Jul 10, 2026
…-wide for confirmed miners (#4546)

* fix(review): make submitter-reputation burst/AI-spend defense install-wide for confirmed miners

getSubmitterReputation's burst/low-sample thresholds are scoped
WHERE project = ? AND submitter = ? -- a single repo. A fleet
identity spreading a handful of gate-passing-but-low-value PRs
across dozens of repos in one self-hosted install never accumulates
enough same-repo sample density to read as "burst" or "low"
anywhere, so the reputation defense never fires for it and every one
of its submissions burns full paid AI-review spend indefinitely.

- New getSubmitterReputationAcrossInstall in submitter-reputation.ts:
  the identical quality-weighted signal derivation, scoped by
  review_targets.installation_id instead of project (that column
  already existed, migrations/0050; this adds the supporting index).
- New getEffectiveSubmitterReputation in reputation-wire.ts: the
  per-repo signal, additionally widened to the install-wide view for
  a CONFIRMED official Gittensor miner -- but only when the per-repo
  signal alone doesn't already justify caution, so an ordinary
  contributor or an already-flagged submitter pays no extra lookup.
- Extracted the miner-identity check (shared with #4512) into
  src/gittensor/miner-detection-cache.ts so both this module and
  unlinked-issue-guardrail.ts can use it without a circular import
  through processors.ts.
- Wired into both call sites: shouldSkipAiForReputation (the main
  AI-spend gate) and the vision-review reputation check.

Fixes #4513

* fix: renumber migration 0130 -> 0131 (0130 was claimed by #4538, already merged to main)

* fix(test): account for getEffectiveSubmitterReputation's miner-identity check in visual-vision tests

runVisualVisionForAdvisory now resolves reputation via getEffectiveSubmitterReputation
(#4513), which checks confirmed-official-miner identity (a fetch to
api.gittensor.io/miners) whenever the submitter's per-repo signal is
neutral -- a real, intentional behavior change this test file's existing
"no network calls at all" assertions didn't account for. Stub that one
identity check to resolve cleanly and assert precisely that no OTHER
(BYOK/vision-spend) network call happens, rather than asserting zero fetch
calls outright.

* fix(db): renumber migration 0131 -> 0133 (0131/0132 claimed by merged PRs)

origin/main has since merged #4545 (0131_screenshot_table_gate_matrix.sql)
and #4554 (0132_impact_map_query_cache.sql, itself a collision fix); rebase
onto current main and take the next free number.

* fix(db): renumber migration 0133 -> 0134 (0133 claimed by merged PR #4556)

Rebase onto current main and take the next free number now that
migrations/0133_screenshot_table_gate_skill_link.sql (from #4556) occupies
the number this branch previously took.

* fix(db): renumber migration 0134 -> 0139 (0134 has a pre-existing 3-way collision on main)

main currently has three DIFFERENT already-merged files at 0134
(#4549/#4558/#4563) -- tracked separately as its own fix (PR #4577, not yet
merged). Since that fix already claims through 0138, take 0139 here to
avoid colliding with it once it lands; this branch's own
db:migrations:check will stay red until #4577 merges (unrelated to this
branch's own changes), and will need one more rebase afterward.

* test(review): close 2 coverage gaps surfaced by the #4514 rebase-merge

getEffectiveSubmitterReputation's getRepository().catch() needed a real
read-failure test (added); its isConfirmedOfficialMiner().catch() is
unreachable (that function already catches every internal failure point
itself) and getSubmitterReputationAcrossInstall's results ?? [] fallback is
the same "D1 always populates results" case already v8-ignored elsewhere in
this codebase -- both marked accordingly.

* fix(review): isolate #4507's reputation-single-read invariant from cross-test miner-cache pollution

Several earlier tests in this file cache "contributor" as a confirmed official
Gittensor miner (5-min TTL) in the shared per-file D1 instance. That collided
with #4513's new install-wide reputation widening, which correctly detects the
stale cache and adds a 4th reputation-scan D1 read for a submitter this test
never intended to be a miner. Use a submitter login unique to this test instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(review): add a submission-cadence/inter-arrival-time signal to the reputation and slop models to flag machine-paced submitters

1 participant