Skip to content

fix(scoring): apply penalty label multipliers instead of flooring to 1 - #1038

Merged
JSONbored merged 3 commits into
JSONbored:mainfrom
bohdansolovie:fix/apply-penalty-label-multipliers
Jun 22, 2026
Merged

fix(scoring): apply penalty label multipliers instead of flooring to 1#1038
JSONbored merged 3 commits into
JSONbored:mainfrom
bohdansolovie:fix/apply-penalty-label-multipliers

Conversation

@bohdansolovie

Copy link
Copy Markdown
Contributor

Closes #994

Summary

selectLabelMultiplier computed the per-PR label multiplier as Math.max(fallback || 1, ...matchedMultipliers). Because the neutral fallback is always >= 1, any configured penalty label multiplier below 1 (e.g. refactor: 0.5, docs: 0.3) was clamped back up to 1 and never applied. Only bonus multipliers (>= 1) took effect, so estimatedMergedScore was inflated for exactly the low-value-label PRs penalty multipliers exist to dampen.

This PR applies matched multipliers directly and falls back to the neutral default only when no configured label matches. The existing "highest matched multiplier wins" tie-break is preserved (bonus + penalty on the same PR is unchanged).

Root cause

// src/scoring/preview.ts — before
return Math.max(
  fallback || 1, // floor is always >= 1
  ...Object.entries(multipliers).flatMap(([label, m]) => (normalized.has(label.toLowerCase()) ? [m] : [])),
);

Verified against the repo fixture { bug: 1.2, refactor: 0.5 }, fallback 1:

Labels Intended multiplier Before (bug) After (fix)
["refactor"] 0.5 1 (2× over-reward) 0.5
["docs"] (0.3 config) 0.3 1 (~3.3× over-reward) 0.3
["bug", "refactor"] highest match wins 1.2 (0.5 dropped) 1.2
["bug"] 1.2 1.2 1.2
["feature"] (no match) 1 1 1

Changes

  • src/scoring/preview.ts — fix selectLabelMultiplier so sub-1 matched multipliers are not floored against the >= 1 fallback.
  • src/services/score-breakdown.ts — surface penalty label multipliers as a "reduced" band with an accurate summary/lever in score breakdown explanations.
  • test/unit/scoring.test.ts — regression coverage for penalty-only, unmatched, bonus+penalty, custom defaultLabelMultiplier, and merged-score scaling.
  • test/unit/score-breakdown.test.ts — assert penalty labels render as band: "reduced".

Behavior changes only when matched labels are all < 1 (previously wrongly floored to 1). Unmatched labels and bonus multipliers are unchanged.

Reachability

labelMultipliers is synced from the upstream registry per repo and input.labels is caller-supplied to buildScorePreview via MCP score_preview and score-preview / explain-breakdown routes. Any repo with a penalty label configured was affected on every preview for PRs carrying that label.

API / OpenAPI / MCP contract

No schema changes. Score preview output semantics change for penalty-labeled PRs: scoreEstimate.labelMultiplier and estimatedMergedScore now reflect configured sub-1 multipliers instead of silently flooring to 1.

Migration / deploy / secrets

None.

Security / privacy

None. Scoring math only; no auth, cookie, CORS, GitHub App output, or contributor evidence changes.

Validation

Run from repo root (Node >= 22):

git diff --check
npm run actionlint
npm run typecheck
npm run test:coverage
npm run test:workers
npm run build:mcp
npm run test:mcp-pack
npm run ui:openapi:check
npm run ui:version-audit
npm run ui:lint
npm run ui:typecheck
npm run ui:test
npm run ui:build

Equivalent combined gate:

npm run test:ci

All gates passed locally on branch fix/apply-penalty-label-multipliers.

@bohdansolovie
bohdansolovie requested a review from JSONbored as a code owner June 22, 2026 13:42
@dosubot dosubot Bot added the size:S label Jun 22, 2026
@ghost ghost added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jun 22, 2026
@ghost

ghost commented Jun 22, 2026

Copy link
Copy Markdown

Note

🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦

💡 Gittensory review — advisory only

0 files · no blockers · readiness 0/100

💡 Advisory only — no action taken

Signal Result Evidence
Code review ✅ No blockers synthesized
Gate result ⚠️ Skipped #1038 is no longer open.

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

@ghost ghost added the gittensory:reviewed label Jun 22, 2026
@ghost

ghost commented Jun 22, 2026

Copy link
Copy Markdown

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

🛠️ Maintainer review — Held for maintainer review

Reviewed 4 changed file(s) — two independent AI reviewers, synthesized. ✅ Safe to merge — both reviewers found no blocking issues.

ℹ️ Correct and all CI green — held for maintainer review.

Merge readiness: ✅ Mergeable · all CI checks green/neutral/skipped.

Summary

The changes correctly adjust label multiplier handling to allow penalty multipliers (<1) instead of flooring to 1, and update the breakdown reporting to reflect a new "reduced" band. Tests cover the new behavior and all CI checks pass. No security or correctness defects are introduced.

💡 Nits — non-blocking, optional

  • Consider adding a brief comment to selectLabelMultiplier explaining the fallback logic for future maintainers.
  • Rename the matched variable to something more descriptive like applicableMultipliers for clarity.
  • Consider adding a comment explaining the rationale behind the change in src/scoring/preview.ts and src/services/score-breakdown.ts for better maintainability.

@ghost ghost added the reviewbot-review label Jun 22, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Held for maintainer (guardrail_area) — correct + CI green

@codecov

codecov Bot commented Jun 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.85%. Comparing base (1ebd641) to head (e4cbf6c).

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1038       +/-   ##
===========================================
+ Coverage   71.76%   96.85%   +25.08%     
===========================================
  Files         141      141               
  Lines       17499    17501        +2     
  Branches     6331     6334        +3     
===========================================
+ Hits        12559    16951     +4392     
+ Misses       3789      104     -3685     
+ Partials     1151      446      -705     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dosubot dosubot Bot added the lgtm label Jun 22, 2026
@JSONbored
JSONbored merged commit 3425026 into JSONbored:main Jun 22, 2026
16 checks passed
@github-actions github-actions Bot mentioned this pull request Jun 22, 2026
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: penalty label multipliers (<1) are floored to 1 and never applied

2 participants