You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Issue #6170 (closed) established that packages/loopover-engine/src has a family of composite-scorer
weight-resolution modules that must all handle an explicit all-zero weight input identically: when a caller
passes every weight as 0, the correct behavior is to preserve the explicit zeros through normalization
(not silently substitute the module's own default blend), so that a downstream "usable weights" stage can
detect the all-zero signal and fall back to an objective-only composite score instead of silently blending
with stale defaults the caller never asked for.
reviewer-consensus-calibration.ts is the reference implementation for both halves of this pattern:
normalizeCompositeWeights (lines 388-409) returns { objectiveAnchor: 0, pairwiseJudge: 0, structuredReviewerConsensus: 0 } on total <= 0, NOT the module's DEFAULT_COMPOSITE_WEIGHTS.
buildRepoRewardRisk's composite-score assembly (lines 566-583) then takes those rawWeights, builds a
second usableWeights/total pass (zeroing out any component whose OWN signal is unavailable, e.g. pairwiseJudgeScore === null), and falls back to { objectiveAnchor: 1, pairwiseJudge: 0, structuredReviewerConsensus: 0 } — an explicit objective-only score — only when that final usable total
is <= 0.
#6170 backported half 1 (and a related malformed-repo-row fix) to gate-verdict-calibration.ts and finding-severity-calibration.ts, explicitly naming "three sibling calibration modules" and stating reviewer-consensus-calibration.ts is "already correct and is the reference implementation." Both named
siblings are now confirmed fixed (gate-verdict-calibration.ts:322, finding-severity-calibration.ts:369
both return explicit zeros; both also have the matching second-stage objective-only fallback at their own
composite-assembly call sites).
A FOURTH module with the identical composite-scorer weight-resolution shape exists in the same package and
was not part of #6170's audit scope at all: pairwise-calibration.ts. Its normalizePairwiseWeights (lines
58-72) has the exact same un-fixed bug #6170 found in the other two:
Worse, pairwise-calibration.ts is also missing half 2 of the pattern entirely: computePairwiseCalibrationScore
(lines 123-154) uses normalizePairwiseWeights(input.weights)'s result directly as weights with no second
"usable weights" fallback stage at all. This means a mechanical copy of #6170's fix (just deleting the DEFAULT_PAIRWISE_WEIGHTS fallback) would make things WORSE here, not better: with pairwiseJudgeScore
non-null and truly-zero weights passed through, compositeScore would become objectiveAnchorScore * 0 + pairwiseJudgeScore * 0 = 0 — a real, valid score silently collapsed to zero instead of falling back to the
objective-anchor score alone the way the reference implementation does.
Requirements
normalizePairwiseWeights must return the explicit zeroed weights ({ objectiveAnchor: 0, pairwiseJudge: 0 }) when total <= 0, matching all three already-fixed siblings — not DEFAULT_PAIRWISE_WEIGHTS.
computePairwiseCalibrationScore must gain the missing second-stage fallback: compute a usableWeights
total from the (possibly explicit-zero) normalized weights and, when that usable total is <= 0, use an
objective-anchor-only weight set ({ objectiveAnchor: 1, pairwiseJudge: 0 }) for the actual composite-score
formula — mirroring reviewer-consensus-calibration.ts's buildRepoRewardRisk assembly (lines 566-583)
exactly, adapted to this module's two-weight (not three-weight) shape.
Do not change reviewer-consensus-calibration.ts, gate-verdict-calibration.ts, or finding-severity-calibration.ts — they are already correct.
Deliverables
normalizePairwiseWeights: explicit-zero passthrough fix (matching the three siblings).
computePairwiseCalibrationScore: add the missing objective-only fallback stage when the usable
weights total is <= 0.
Regression test: weights: { objectiveAnchor: 0, pairwiseJudge: 0 } with a real (non-null) pairwiseJudgeScore present — asserts compositeScore === objectiveAnchorScore (the objective-only
fallback), not 0.
Existing test at packages/loopover-engine/test/pairwise-calibration.test.ts:146 (NaN/-1 recovers to
the 50/50 default) continues to pass unchanged.
Test Coverage Requirements
99%+ Codecov patch coverage (branch-counted) on both changed functions. The explicit-all-zero-weights
regression test above is the one that actually proves this issue is fixed — without it, both the
old-silent-default bug and a naive "return zeros with no second-stage fallback" incomplete fix would look
identical to the existing test suite.
Expected Outcome
All four sibling calibration modules (gate-verdict-calibration.ts, finding-severity-calibration.ts, reviewer-consensus-calibration.ts, pairwise-calibration.ts) handle an explicit all-zero weight input
identically — a real, valid signal falls back to an objective-only score, never to a silently-substituted
default blend the caller never asked for, and never to a spuriously-zeroed composite score.
packages/loopover-engine/src/reviewer-consensus-calibration.ts (normalizeCompositeWeights lines
388-409; the composite assembly at lines 557-586) — the reference implementation for both halves
Context
Issue #6170 (closed) established that
packages/loopover-engine/srchas a family of composite-scorerweight-resolution modules that must all handle an explicit all-zero weight input identically: when a caller
passes every weight as
0, the correct behavior is to preserve the explicit zeros through normalization(not silently substitute the module's own default blend), so that a downstream "usable weights" stage can
detect the all-zero signal and fall back to an objective-only composite score instead of silently blending
with stale defaults the caller never asked for.
reviewer-consensus-calibration.tsis the reference implementation for both halves of this pattern:normalizeCompositeWeights(lines 388-409) returns{ objectiveAnchor: 0, pairwiseJudge: 0, structuredReviewerConsensus: 0 }ontotal <= 0, NOT the module'sDEFAULT_COMPOSITE_WEIGHTS.buildRepoRewardRisk's composite-score assembly (lines 566-583) then takes thoserawWeights, builds asecond
usableWeights/totalpass (zeroing out any component whose OWN signal is unavailable, e.g.pairwiseJudgeScore === null), and falls back to{ objectiveAnchor: 1, pairwiseJudge: 0, structuredReviewerConsensus: 0 }— an explicit objective-only score — only when that final usable totalis
<= 0.#6170 backported half 1 (and a related malformed-repo-row fix) to
gate-verdict-calibration.tsandfinding-severity-calibration.ts, explicitly naming "three sibling calibration modules" and statingreviewer-consensus-calibration.tsis "already correct and is the reference implementation." Both namedsiblings are now confirmed fixed (
gate-verdict-calibration.ts:322,finding-severity-calibration.ts:369both return explicit zeros; both also have the matching second-stage objective-only fallback at their own
composite-assembly call sites).
A FOURTH module with the identical composite-scorer weight-resolution shape exists in the same package and
was not part of #6170's audit scope at all:
pairwise-calibration.ts. ItsnormalizePairwiseWeights(lines58-72) has the exact same un-fixed bug #6170 found in the other two:
Worse,
pairwise-calibration.tsis also missing half 2 of the pattern entirely:computePairwiseCalibrationScore(lines 123-154) uses
normalizePairwiseWeights(input.weights)'s result directly asweightswith no second"usable weights" fallback stage at all. This means a mechanical copy of #6170's fix (just deleting the
DEFAULT_PAIRWISE_WEIGHTSfallback) would make things WORSE here, not better: withpairwiseJudgeScorenon-null and truly-zero weights passed through,
compositeScorewould becomeobjectiveAnchorScore * 0 + pairwiseJudgeScore * 0 = 0— a real, valid score silently collapsed to zero instead of falling back to theobjective-anchor score alone the way the reference implementation does.
Requirements
normalizePairwiseWeightsmust return the explicit zeroed weights ({ objectiveAnchor: 0, pairwiseJudge: 0 }) whentotal <= 0, matching all three already-fixed siblings — notDEFAULT_PAIRWISE_WEIGHTS.computePairwiseCalibrationScoremust gain the missing second-stage fallback: compute ausableWeightstotal from the (possibly explicit-zero) normalized weights and, when that usable total is
<= 0, use anobjective-anchor-only weight set (
{ objectiveAnchor: 1, pairwiseJudge: 0 }) for the actual composite-scoreformula — mirroring
reviewer-consensus-calibration.ts'sbuildRepoRewardRiskassembly (lines 566-583)exactly, adapted to this module's two-weight (not three-weight) shape.
NaN, negative numbers) must continue to recover toDEFAULT_PAIRWISE_WEIGHTSastoday — this fix only changes the behavior for a caller's EXPLICIT, well-formed all-zero input, exactly as
fix(engine): backport reviewer-consensus-calibration.ts's zero-weight and malformed-repo handling to its two older siblings #6170 scoped its own two fixes.
reviewer-consensus-calibration.ts,gate-verdict-calibration.ts, orfinding-severity-calibration.ts— they are already correct.Deliverables
normalizePairwiseWeights: explicit-zero passthrough fix (matching the three siblings).computePairwiseCalibrationScore: add the missing objective-only fallback stage when the usableweights total is
<= 0.weights: { objectiveAnchor: 0, pairwiseJudge: 0 }with a real (non-null)pairwiseJudgeScorepresent — assertscompositeScore === objectiveAnchorScore(the objective-onlyfallback), not
0.packages/loopover-engine/test/pairwise-calibration.test.ts:146(NaN/-1 recovers tothe 50/50 default) continues to pass unchanged.
Test Coverage Requirements
99%+ Codecov patch coverage (branch-counted) on both changed functions. The explicit-all-zero-weights
regression test above is the one that actually proves this issue is fixed — without it, both the
old-silent-default bug and a naive "return zeros with no second-stage fallback" incomplete fix would look
identical to the existing test suite.
Expected Outcome
All four sibling calibration modules (
gate-verdict-calibration.ts,finding-severity-calibration.ts,reviewer-consensus-calibration.ts,pairwise-calibration.ts) handle an explicit all-zero weight inputidentically — a real, valid signal falls back to an objective-only score, never to a silently-substituted
default blend the caller never asked for, and never to a spuriously-zeroed composite score.
Links & Resources
packages/loopover-engine/src/pairwise-calibration.ts(normalizePairwiseWeightslines 58-72,computePairwiseCalibrationScorelines 123-154)packages/loopover-engine/src/reviewer-consensus-calibration.ts(normalizeCompositeWeightslines388-409; the composite assembly at lines 557-586) — the reference implementation for both halves
packages/loopover-engine/src/gate-verdict-calibration.ts:322andpackages/loopover-engine/src/finding-severity-calibration.ts:369— the two siblings fix(engine): backport reviewer-consensus-calibration.ts's zero-weight and malformed-repo handling to its two older siblings #6170 already fixedpackages/loopover-engine/test/pairwise-calibration.test.ts:146— the existing (different-intent) test topreserve