Skip to content

fix(engine): converge gate-verdict + finding-severity calibration onto reviewer-consensus's zero-weight + malformed-repo handling - #6251

Merged
loopover-orb[bot] merged 2 commits into
JSONbored:mainfrom
real-venus:fix/engine-calibration-converge-zero-weight-malformed-repo
Jul 16, 2026
Merged

fix(engine): converge gate-verdict + finding-severity calibration onto reviewer-consensus's zero-weight + malformed-repo handling#6251
loopover-orb[bot] merged 2 commits into
JSONbored:mainfrom
real-venus:fix/engine-calibration-converge-zero-weight-malformed-repo

Conversation

@real-venus

Copy link
Copy Markdown
Contributor

Summary

Converges the two older calibration modules onto the newest/correct reviewer-consensus-calibration.ts in two ways it never got backported:

  1. Zero-weight handling (gate-verdict-calibration.ts, finding-severity-calibration.ts): both did if (total <= 0) return DEFAULT_COMPOSITE_WEIGHTS, silently restoring the 45/35/20 blend when a caller explicitly zeroes every component. They now return the explicit zeroed weights, so an all-zero config reaches the objective-only fallback -- matching reviewer-consensus.
  2. Malformed-repo preservation (gate-verdict-calibration.ts): the rejected-row sanitizer used normalizeRepoFullName(row.repoFullName) alone (returns null for an already-rejected invalid_repo row -> drops it). It now uses normalizeRepoFullName(...) ?? normalizeId(...) (matching the other two files), preserving the raw string.

reviewer-consensus-calibration.ts is unchanged -- the reference for both fixes.

Tests

  • Engine node:test regressions in packages/loopover-engine/test/gate-verdict-calibration.test.ts (zero-weight + invalid_repo-survives) and .../finding-severity-calibration.test.ts (zero-weight), mirroring reviewer-consensus's equivalents.
  • A root vitest suite (test/unit/engine-calibration-convergence.test.ts) covers the changed engine-src branches for codecov/patch (the calibration modules are otherwise only node:test-covered, which does not feed Codecov). It exercises both branches of each changed if (total <= 0) line (all-zero -> objective-only, and non-zero -> normalized) and all ternary/?? legs of the malformed-repo line (valid / invalid bad / non-string). Verified 100% branch coverage on the three changed lines locally.

Validation

Safety

  • No secrets/private terms. Behavior change is scoped to malformed/all-zero input, converging two calibrators onto the already-correct third; legitimate-input scoring is unchanged.

Closes #6170

@real-venus
real-venus requested a review from JSONbored as a code owner July 15, 2026 23:45
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.60%. Comparing base (e9977ad) to head (696d111).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6251      +/-   ##
==========================================
+ Coverage   95.35%   95.60%   +0.24%     
==========================================
  Files         598      598              
  Lines       47127    47127              
  Branches    14994    14994              
==========================================
+ Hits        44940    45057     +117     
+ Misses       1463     1288     -175     
- Partials      724      782      +58     
Flag Coverage Δ
shard-1 44.04% <0.00%> (ø)
shard-2 36.59% <0.00%> (ø)
shard-3 32.42% <0.00%> (+0.08%) ⬆️
shard-4 34.52% <100.00%> (+0.16%) ⬆️
shard-5 31.79% <0.00%> (ø)
shard-6 44.91% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...oopover-engine/src/finding-severity-calibration.ts 32.59% <100.00%> (+30.38%) ⬆️
...es/loopover-engine/src/gate-verdict-calibration.ts 39.05% <100.00%> (+36.68%) ⬆️

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 16, 2026
@loopover-orb

loopover-orb Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Tip

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-16 00:03:29 UTC

5 files · 1 AI reviewer · no blockers · readiness 80/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This PR backports two behaviors from reviewer-consensus-calibration.ts into gate-verdict-calibration.ts and finding-severity-calibration.ts: (1) an explicitly all-zero weight config now returns the zeroed weights object instead of silently substituting DEFAULT_COMPOSITE_WEIGHTS, and (2) a rejected row with an unparseable-but-string repoFullName now falls back to normalizeId instead of being dropped. Both changes are minimal, mechanical, and directly mirror the already-correct sibling module, and the added tests (node:test + a new vitest suite for Codecov) exercise both branches of the changed `if (total <= 0)` lines and all three legs of the `?? normalizeId` ternary (valid, invalid-string, non-string). The PR is explicitly tied to #6170 and doesn't introduce speculative scope beyond the stated convergence.

Nits — 5 non-blocking
  • The vitest test at test/unit/engine-calibration-convergence.test.ts duplicates assertions already present in the engine's own node:test files almost verbatim -- consider extracting the shared fixture (e.g. the gateVerdicts/findingSeverity sample objects) instead of copy-pasting across three test files.
  • The inline comments explaining the zero-weight fallback are repeated nearly word-for-word in both gate-verdict-calibration.ts:319 and finding-severity-calibration.ts:366; a shared JSDoc reference to reviewer-consensus-calibration.ts would avoid drift if the rationale changes.
  • Confirm the composite scorer (not shown in this diff) actually maps an all-zero weights object to the objective-only path for both files, since normalizeCompositeWeights only returns the zeroed struct here -- the fallback-to-objective-only behavior must live elsewhere and isn't visible in this diff.
  • Consider a small shared test-fixture module for the gateVerdicts/findingSeverity sample rows reused across gate-verdict-calibration.test.ts, finding-severity-calibration.test.ts, and the new vitest suite.
  • If reviewer-consensus-calibration.ts's zero-weight/malformed-repo logic ever changes again, add a lint/test guard (e.g. a diff-parity check) so the three modules don't drift a third time.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #6170
Related work ⚠️ 2 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 139 registered-repo PR(s), 75 merged, 21 issue(s).
Contributor context ✅ Confirmed Gittensor contributor real-venus; Gittensor profile; 139 PR(s), 21 issue(s).
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Addressed
The diff makes both required changes: gate-verdict-calibration.ts and finding-severity-calibration.ts now return explicit zeroed weights instead of DEFAULT_COMPOSITE_WEIGHTS on total<=0, and gate-verdict-calibration.ts's sanitizer now uses normalizeRepoFullName(...) ?? normalizeId(...) to preserve malformed repo rows, exactly matching the reference implementation and requirements.

Review context
Contributor next steps
  • Start here: Review top overlaps.
  • Then work through the remaining 2 steps in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoopOver approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit e30a268 into JSONbored:main Jul 16, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(engine): backport reviewer-consensus-calibration.ts's zero-weight and malformed-repo handling to its two older siblings

1 participant