Skip to content

fix(scoring): exclude maintainer-generated test commits from contributor scoring #4201

Description

@JSONbored

Part of #4189. Depends on #4197 (the commit-delivery mode is what introduces the actual commit needing exclusion).

Context

#1972's "no-cloud-write boundary" exists to protect contributor/miner scoring integrity — a miner's score must reflect their own work. #4197 deliberately creates a narrow, maintainer-only exception to that boundary (a maintainer, not the contributor, authorizes gittensory to write a commit onto the PR branch). That exception is only safe if the resulting commit can never be silently counted as part of the contributor's scored contribution — otherwise a maintainer generating tests would inadvertently inflate the PR author's Gittensor points for code they didn't write, which is the exact failure mode the boundary was built to prevent, just via a different path.

This issue is a verification-and-fix, not a known bug: whatever computes a PR's scored diff size/contribution weight for Gittensor points needs to be checked for whether it already excludes bot/app-authored commits (e.g. it might already be diff-by-commit-author rather than raw base..head diff, in which case this issue is a fast confirmation) or whether it needs a new exclusion rule added.

Requirements

  • Identify exactly where contribution size/diff weight is computed for Gittensor scoring purposes (candidates: src/scoring/, src/gittensor/, wherever PR review outcomes feed the points model) and determine whether it measures base...head diff wholesale or sums only commits authored by the PR's actual contributor.
  • If it already excludes non-author commits: add a regression test proving a gittensory-authored test commit does NOT change the PR's computed score, and close this out as verified, not just assumed.
  • If it does NOT already exclude them: add commit-author-based exclusion (the GitHub App's own bot identity is a stable, verifiable signal — same identity already used to post the unified review comment) so a maintainer-added commit is subtracted from whatever diff the scoring model measures.
  • Whatever mechanism is chosen must also apply retroactively-safely — i.e. not require re-scoring already-merged historical PRs, only affect PRs scored from this change forward.
  • Document the finding (excluded-already vs. needed-a-fix) clearly in the implementing PR, since this is exactly the kind of boundary-integrity detail that should be traceable later.

Deliverables

  • A definitive answer, backed by a passing test, on whether gittensory-authored commits already are or now are excluded from contributor scoring.
  • If a fix was needed: the exclusion mechanism, plus a regression test using a synthetic bot-authored commit.

Expected outcome

A maintainer using @gittensory generate-tests with commit delivery never inflates the PR author's own scored contribution — the maintainer-only exception to the no-cloud-write boundary stays narrow and auditable in both directions (who can trigger it, and what it does to scoring), not just the trigger side.

Resources / examples

  • #1972 — the original boundary rationale ("Generating tests server-side would break the contributor boundary")
  • src/scoring/model.ts, src/scoring/preview.ts, src/scoring/pending-pr-scenarios.ts (starting points to trace the actual scoring computation — not yet confirmed which one owns diff-size measurement)

Effort

S-M — likely a fast confirmation if scoring already keys off commit authorship; M if a new exclusion needs to be built.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.orbGittensory Orb related - maintainer self-hosting analytics.roadmapOn the Wave-2 agent-layer roadmap board (project 9)

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions