[Bug]: penalty label multipliers (< 1) are silently floored to 1 and never applied — over-rewards low-value PRs
Summary
selectLabelMultiplier computes the per-PR label multiplier as
Math.max(fallback || 1, ...matchedMultipliers). Because the neutral fallback is
always >= 1, any configured penalty label multiplier below 1 (e.g.
refactor: 0.5, docs: 0.3) is clamped back up to 1 and never applied. Only
bonus multipliers (>= 1) ever take effect. A maintainer who configures
penalty labels to dampen low-value PRs gets them silently ignored, and the
contributor's token score / merged-PR estimate is inflated by the dropped
penalty.
The product clearly intends sub-1 multipliers to work: the registry schema allows
any number, the upstream registry sync parses label_multipliers verbatim, the
ruleset treats labelMultipliers as first-class scoreability config — and the
codebase's own scoring fixture ships refactor: 0.5.
Evidence
// src/scoring/preview.ts:736
function selectLabelMultiplier(labels: string[], multipliers: Record<string, number>, fallback: number): number {
const normalized = new Set(labels.map((label) => label.toLowerCase()));
return Math.max(
fallback || 1, // floor is always >= 1
...Object.entries(multipliers).flatMap(([label, multiplier]) => (normalized.has(label.toLowerCase()) ? [multiplier] : [])),
);
}
The result multiplies the base score:
// src/scoring/preview.ts:320 + :353
const labelMultiplier = selectLabelMultiplier(input.labels ?? [], config?.labelMultipliers ?? {}, config?.defaultLabelMultiplier ?? 1);
...
baseScore * labelMultiplier * issueMultiplier * credibilityMultiplier * reviewPenaltyMultiplier * openPrMultiplier * openIssueMultiplier * timeDecayMultiplier
Verified by executing the exact logic against the config { bug: 1.2, refactor: 0.5, docs: 0.3 }, fallback 1:
| Labels |
Intended multiplier |
Actual |
["refactor"] |
0.5 |
1 |
["docs"] |
0.3 |
1 |
["bug", "refactor"] |
(penalty present) |
1.2 (0.5 dropped) |
["bug"] |
1.2 |
1.2 ✓ |
["feature"] (no match) |
1 |
1 ✓ |
So a refactor PR that should score baseScore * 0.5 instead scores
baseScore * 1 — 2× the intended reward; a docs PR at 0.3 scores ~3.3×.
The schema imposes no lower bound (so the config is valid):
// src/openapi/schemas.ts:40
labelMultipliers: z.record(z.string(), z.number()),
Reachability
Fully reachable from the public scoring surface. labelMultipliers is synced from
the upstream registry per repo (src/registry/normalize.ts, src/upstream/ruleset.ts:573),
and input.labels is caller-supplied to buildScorePreview via the MCP
score_preview tool and the score-preview/explain-breakdown routes. Any repo whose
registry config sets a penalty label multiplier gets it silently dropped on every
score preview/breakdown for a PR carrying that label.
Suggested fix
Apply the matched multiplier(s) when any label matches, and fall back to the
neutral default only when nothing matches — don't Math.max against the >= 1
floor. Preserve the existing "highest multiplier among matched labels wins"
tie-break (so a bonus+penalty PR is unchanged), which removes ONLY the erroneous
flooring of a lone/all-penalty match:
function selectLabelMultiplier(labels: string[], multipliers: Record<string, number>, fallback: number): number {
const normalized = new Set(labels.map((label) => label.toLowerCase()));
const matched = Object.entries(multipliers).flatMap(([label, multiplier]) =>
normalized.has(label.toLowerCase()) ? [multiplier] : [],
);
return matched.length > 0 ? Math.max(...matched) : (fallback || 1);
}
(If the intended multi-label rule is "strictest penalty wins," use Math.min(...matched)
instead — but the key fix either way is that the matched set must NOT be maxed
against the >= 1 fallback, so a sub-1 penalty actually bites.)
Test status
Not covered — and the suite has a telling decoy: test/unit/scoring.test.ts
defines labelMultipliers: { bug: 1.2, refactor: 0.5 } in the repo fixture but
never feeds the refactor label into a preview. Every label assertion uses a
bonus or empty config: bug → 1.2, empty-config fallback → 1,
defaultLabelMultiplier 1.05. So the refactor: 0.5 penalty entry is unexercised
dead decoration and the flooring ships green. A regression test should assert a
PR labeled ["refactor"] yields scoreEstimate.labelMultiplier === 0.5 (and that
the merged-score estimate is correspondingly halved).
Confidence note
High. The flooring is proven by executing the exact formula on the config the
codebase itself ships, the multiplier demonstrably scales the score, sub-1
multipliers are a first-class synced config (schema + registry + ruleset), and the
penalty case is provably untested. It is a core scoring-correctness defect that
systematically over-rewards exactly the low-value-label PRs the penalty exists to
dampen.
Distinct from prior reports
Core scoring math, unrelated to the fractional-underscore constant parse (#992),
the suspended-installation scope (#953), predicted-gate, gate-403, or BYOK.
[Bug]: penalty label multipliers (< 1) are silently floored to 1 and never applied — over-rewards low-value PRs
Summary
selectLabelMultipliercomputes the per-PR label multiplier asMath.max(fallback || 1, ...matchedMultipliers). Because the neutral fallback isalways
>= 1, any configured penalty label multiplier below 1 (e.g.refactor: 0.5,docs: 0.3) is clamped back up to 1 and never applied. Onlybonus multipliers (
>= 1) ever take effect. A maintainer who configurespenalty labels to dampen low-value PRs gets them silently ignored, and the
contributor's token score / merged-PR estimate is inflated by the dropped
penalty.
The product clearly intends sub-1 multipliers to work: the registry schema allows
any number, the upstream registry sync parses
label_multipliersverbatim, theruleset treats
labelMultipliersas first-class scoreability config — and thecodebase's own scoring fixture ships
refactor: 0.5.Evidence
The result multiplies the base score:
Verified by executing the exact logic against the config
{ bug: 1.2, refactor: 0.5, docs: 0.3 }, fallback 1:["refactor"]["docs"]["bug", "refactor"]["bug"]["feature"](no match)So a
refactorPR that should scorebaseScore * 0.5instead scoresbaseScore * 1— 2× the intended reward; adocsPR at0.3scores ~3.3×.The schema imposes no lower bound (so the config is valid):
Reachability
Fully reachable from the public scoring surface.
labelMultipliersis synced fromthe upstream registry per repo (
src/registry/normalize.ts,src/upstream/ruleset.ts:573),and
input.labelsis caller-supplied tobuildScorePreviewvia the MCPscore_previewtool and the score-preview/explain-breakdown routes. Any repo whoseregistry config sets a penalty label multiplier gets it silently dropped on every
score preview/breakdown for a PR carrying that label.
Suggested fix
Apply the matched multiplier(s) when any label matches, and fall back to the
neutral default only when nothing matches — don't
Math.maxagainst the>= 1floor. Preserve the existing "highest multiplier among matched labels wins"
tie-break (so a bonus+penalty PR is unchanged), which removes ONLY the erroneous
flooring of a lone/all-penalty match:
(If the intended multi-label rule is "strictest penalty wins," use
Math.min(...matched)instead — but the key fix either way is that the matched set must NOT be maxed
against the
>= 1fallback, so a sub-1 penalty actually bites.)Test status
Not covered — and the suite has a telling decoy:
test/unit/scoring.test.tsdefines
labelMultipliers: { bug: 1.2, refactor: 0.5 }in the repo fixture butnever feeds the
refactorlabel into a preview. Every label assertion uses abonus or empty config:
bug → 1.2, empty-config fallback→ 1,defaultLabelMultiplier 1.05. So therefactor: 0.5penalty entry is unexerciseddead decoration and the flooring ships green. A regression test should assert a
PR labeled
["refactor"]yieldsscoreEstimate.labelMultiplier === 0.5(and thatthe merged-score estimate is correspondingly halved).
Confidence note
High. The flooring is proven by executing the exact formula on the config the
codebase itself ships, the multiplier demonstrably scales the score, sub-1
multipliers are a first-class synced config (schema + registry + ruleset), and the
penalty case is provably untested. It is a core scoring-correctness defect that
systematically over-rewards exactly the low-value-label PRs the penalty exists to
dampen.
Distinct from prior reports
Core scoring math, unrelated to the fractional-underscore constant parse (#992),
the suspended-installation scope (#953), predicted-gate, gate-403, or BYOK.