Skip to content

feat(agent): deterministic local scorer MCP tool (#782) - #847

Merged
JSONbored merged 1 commit into
mainfrom
feat/agent-local-scorer-782
Jun 18, 2026
Merged

feat(agent): deterministic local scorer MCP tool (#782)#847
JSONbored merged 1 commit into
mainfrom
feat/agent-local-scorer-782

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Closes #782

What & why

Phase 2 (#770). Closes the "miner runs the gittensor-root scorer by hand" gap. A new MCP tool gittensory_run_local_scorer(changed_files, validation) returns token scores the miner feeds straight back as the localScorer field of gittensory_preview_local_pr_score / the analyze tools — flipping the score preview into external_command mode (off metadata-only).

Per the agreed contract: deterministic replicate, not a binary — surface-agnostic (works in the stdio package and the hosted Worker) and boundary-safe.

How

  • src/signals/local-scorer.ts — pure computeLocalScorerTokens: classifies source / test / non-code from changed-file metadata (paths + line counts, never source content). Mirrors buildScorePreview's classification (isCodeFile already excludes tests, so the three buckets are disjoint); binary files carry no token value; a failed local validation is surfaced as a warning without changing the scores.
  • src/mcp/server.ts — register the tool + a pure runLocalScorer method (no repo/contributor access — it only computes over the caller's own diff stats). Extracted the shared changedFile / validation zod schemas into named consts so the analyze shape and the new tool share one definition.

Boundary

The tool takes metadata only (file paths + additions/deletions + validation status) — never source content — so the no-upload boundary holds and it runs in both MCP surfaces.

Tests

The pure scorer (classification, binary drop, sourceLines fallback + floor, validation warning) and the MCP tool round-trip (token scores + usage hint + no reward/wallet leakage). New code 100% covered; MCP discovery/output-schema/telemetry meta-tests green; full suite green (2096 passed).

Phase 2: closes the 'miner runs the gittensor-root scorer by hand' gap. New MCP
tool gittensory_run_local_scorer(changed_files, validation) returns token scores
the miner feeds back as the localScorer field of the score-preview / analyze
tools (external_command mode → off metadata-only).

- src/signals/local-scorer.ts: pure computeLocalScorerTokens — classifies
  source / test / non-code from changed-file METADATA (paths + line counts,
  never source content, so the no-upload boundary holds and it runs in every
  surface). Mirrors buildScorePreview's classification (isCodeFile excludes
  tests, so the buckets are disjoint); binary files carry no token value; a
  failed local validation is surfaced as a warning without changing the scores.
- src/mcp/server.ts: register the tool + a pure runLocalScorer method (no
  repo/contributor access — it only computes over the caller's own diff stats).
  Extracted the shared changedFile/validation zod schemas into named consts so
  the analyze shape and the scorer tool reuse one definition.

Tests: the pure scorer (classification, binary drop, sourceLines fallback +
floor, validation warning) and the MCP tool round-trip (token scores + usage
hint + no reward/wallet leakage). New code 100% covered; full suite green (2096).
@dosubot dosubot Bot added the size:L label Jun 18, 2026
@ghost

ghost commented Jun 18, 2026

Copy link
Copy Markdown

Note

Gittensory Gate skipped

PR closed before full evaluation. No late first comment was created.

Signal Result Evidence Action
Gate result ⚠️ Skipped #847 is no longer open. No action.

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

@ghost ghost added the gittensory:reviewed label Jun 18, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.66%. Comparing base (80a1f48) to head (1316419).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #847   +/-   ##
=======================================
  Coverage   96.66%   96.66%           
=======================================
  Files         105      106    +1     
  Lines       14476    14495   +19     
  Branches     5266     5269    +3     
=======================================
+ Hits        13993    14012   +19     
  Misses        105      105           
  Partials      378      378           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ghost

ghost commented Jun 18, 2026

Copy link
Copy Markdown

gittensory · advisory review

Reviewed 4 changed file(s) — two independent AI reviewers.

Changed files: src/mcp/server.ts, src/signals/local-scorer.ts, test/unit/local-scorer.test.ts, test/unit/mcp-run-local-scorer.test.ts

Suggested action:Safe to merge — both reviewers found no blocking issues.

Reviewer A · gpt-oss-120b — recommends ✅ merge
The PR introduces a deterministic local token scorer tool, adds the corresponding implementation, registers it in the MCP server, refactors schemas, and provides comprehensive unit tests. The changes respect the no‑upload boundary and avoid leaking forbidden terms.

Suggestions

  • Add a brief comment in the tool registration block referencing the new feat(agent): local gittensor-root scorer MCP wrapper #782 feature for future maintainers.
  • Consider exporting computeLocalScorerTokens from a central signals index if other modules may need it.
  • Update any API documentation to list the new gittensory_run_local_scorer tool.

Worth double-checking

  • Ensure that any downstream consumers expecting changedFiles to be optional still handle the new required min(1) constraint in the tool's input schema.
  • Verify that the cast to Record<string, unknown> in runLocalScorer does not hide type errors in future changes.

Reviewer B · mistral-small-3.1-24b-instruct — recommends ✅ merge
This PR adds a deterministic local scorer tool to the Gittensory MCP server. It includes the necessary schemas, computation logic, and tests to ensure the tool works correctly. The changes are well-documented and follow the project's conventions.

Suggestions

  • Consider adding a brief comment explaining why the sourceLines field is floored at 1 in the computeLocalScorerTokens function.
  • Ensure that the computeLocalScorerTokens function handles edge cases where additions and deletions are not provided.
  • Add a test case for the scenario where changedFiles is an empty array to ensure the function handles it gracefully.

Worth double-checking

  • Verify that the computeLocalScorerTokens function correctly handles all edge cases, especially when additions and deletions are not provided.
  • Ensure that the new tool does not introduce any performance bottlenecks, especially when processing a large number of changed files.
  • Double-check that the new tool adheres to the project's privacy and safety model, particularly regarding the handling of changed-file metadata.

@JSONbored
JSONbored merged commit 2e06410 into main Jun 18, 2026
20 checks passed
@JSONbored
JSONbored deleted the feat/agent-local-scorer-782 branch June 18, 2026 00:04
@github-actions github-actions Bot mentioned this pull request Jun 18, 2026
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(agent): local gittensor-root scorer MCP wrapper

1 participant