Skip to content

reviewer-consensus-calibration.ts (658 lines) is the package's largest branch surface invisible to Codecov #8349

Description

@JSONbored

Context

packages/loopover-engine/src/reviewer-consensus-calibration.ts is the pure engine half of the opt-in
structured reviewer-consensus calibration signal (multi-reviewer agreement, part of the same calibration
family as finding-severity-calibration.ts and gate-verdict-calibration.ts, converged via #6170's
zero-weight/malformed-repo-handling backport). It exports resolveReviewerConsensusCalibrationConfig
(explicit-opt-in config resolution from a .loopover.yml-style manifest, both the miner.calibration.* path
and a top-level alias), ingestReviewerConsensusCalibrationSignals (accept/reject raw signals by opt-in +
validity), computeReviewerConsensusCompositeCalibrationScore (blends objective-anchor, pairwise-judge, and
this structured signal into one composite, normalizing weights and falling back to an objective-only blend
when every weight is zero or unusable), and renderReviewerConsensusCalibrationAuditMarkdown (deterministic,
public-safe Markdown report). At 658 lines it is one of the largest branch surfaces in the package (~128
branches across config resolution, per-signal validation, weight normalization, and score composition). It
is exported from the engine's public barrel and already has a full, passing test suite at
packages/loopover-engine/test/reviewer-consensus-calibration.test.ts (21 scenarios, node --test).

Same failure mode as #6250 (review-enrichment's node:test suite being invisible to Codecov): that engine
package suite doesn't run under root vitest, which is what Codecov's codecov/patch gate reads.
packages/loopover-engine/src/**/*.ts is already listed in vitest.config.ts's coverage.include, so this
is a per-file gap — nothing under root test/** currently exercises this module directly, so it reports as
~0% covered in Codecov despite real, passing tests existing one directory over, and despite being the
package's single largest untested-by-Codecov branch surface.

⚠️ Read this before starting. This issue is test-coverage-only. Do NOT modify
packages/loopover-engine/src/reviewer-consensus-calibration.ts or
packages/loopover-engine/test/reviewer-consensus-calibration.test.ts. The only new artifact this issue
calls for is one new root-level vitest file. A PR that changes the source module's behavior does not
satisfy this issue.

Requirements

  • Create test/unit/reviewer-consensus-calibration.test.ts using describe/it/expect from vitest,
    importing resolveReviewerConsensusCalibrationConfig, ingestReviewerConsensusCalibrationSignals,
    computeReviewerConsensusCompositeCalibrationScore, and renderReviewerConsensusCalibrationAuditMarkdown
    from "../../packages/loopover-engine/src/index" — the barrel — matching the existing sibling pattern in
    test/unit/calibration-dashboard.test.ts.
  • Cover, at minimum, every scenario already exercised in
    packages/loopover-engine/test/reviewer-consensus-calibration.test.ts (read it for the exact
    fixtures/assertions to mirror), including:
    • resolveReviewerConsensusCalibrationConfig: default opted-out with no manifest; the preferred
      miner.calibration.* path vs. the top-level alias path (and preferred-path precedence when both are
      present); a non-boolean-like opt-in value producing a warning and defaulting to false; a
      negative/non-finite weight producing a warning and defaulting; a valid explicit weight overriding the
      default.
    • ingestReviewerConsensusCalibrationSignals: rejects an invalid repoFullName, an invalid/oversized
      replayRunId/run id, a not-opted-in signal, and an empty/all-invalid vote list — each with its correct
      reason code; accepts a valid signal and computes its score correctly.
    • computeReviewerConsensusCompositeCalibrationScore: normal 3-way weighted blend; a null pairwise score
      zeroing that weight's contribution and renormalizing the remaining two; zero accepted structured signals
      zeroing that weight's contribution; the explicit-all-zero-weights case falling through to the
      objective-anchor-only fallback (per fix(engine): backport reviewer-consensus-calibration.ts's zero-weight and malformed-repo handling to its two older siblings #6170's already-fixed sibling convergence — this must NOT silently
      restore the default blend); both the pre-built-Ingestion-object input form and the raw-signal-array
      input form.
    • renderReviewerConsensusCalibrationAuditMarkdown: the empty-contributing-repos branch, the
      empty-rejected-rows branch, and the populated-table branch, plus markdown-unsafe characters in a
      repo/run-id field being escaped.
  • Every ??, ternary, and boolean short-circuit across all four exported functions must have both its taken
    and not-taken branch exercised — this is the largest branch surface flagged in this coverage sweep, so
    completeness here matters more than for the package's smaller files.

Deliverables

  • New file test/unit/reviewer-consensus-calibration.test.ts (vitest, imports via the engine barrel).
  • No changes to any file under packages/loopover-engine/src/** or packages/loopover-engine/test/**.

Test Coverage Requirements

packages/loopover-engine/src/reviewer-consensus-calibration.ts is inside coverage.include in
vitest.config.ts and therefore IS subject to the 99% Codecov codecov/patch branch-counted gate — today it
reports ~0% because nothing under root test/** exercises it directly. After this PR, npm run test:coverage
(run unsharded locally) should show this file at or near 100% line/branch coverage.

Expected Outcome

reviewer-consensus-calibration.ts's already-real, already-tested config-resolution, ingestion, and
composite-scoring logic becomes visible to Codecov, so a future PR that changes this file is actually gated
on real coverage instead of a blind spot.

Links & Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions