feat(review): pure config-drift evaluator over the loosenable-knob registry

[RealDiligent]

Summary

• Closes calibration: pure config-drift evaluator — does any alternative setting Pareto-dominate the live value on the trailing corpus? #8212
• The loosening evaluator (evaluateKnobLoosening, src/services/loosening-knobs.ts) answers "can we safely loosen?"; nothing answered the inverse operator question from the calibration: backfill phase 2 — GitHub-truth reversed labels (retro successor scan) + raw-context re-fetch #8170 retro's largest identified wrongness source — stale configuration: "is what is CURRENTLY live still the best-supported setting, in either direction?"
• Adds evaluateKnobDrift(knob, cases, liveValue) exported beside LOOSENABLE_KNOBS, mirroring evaluateKnobLoosening's discipline verbatim (the two named anchors): the knob's own split seed/fraction via splitBacktestCorpus, the same Pareto floor via compareBacktestScores (strictly improved on the visible split AND non-regressed on the deterministic held-out split), the same never-on-noise sample minimums, and the hard minimum no evidence may cross.
• The candidate pool is every registry candidate plus the shipped value (a TIGHTER alternative dominating live is exactly the stale-config signal) minus the live value itself, tried nearest-to-live first — the minimal config change wins, mirroring smallest-step-first; equidistant ties deterministically prefer the tighter value. The report distinguishes direction: "shipped" (checked first — a drifted override should revert), "looser" (informational; duplicates the loosening loop's own proposal), "tighter" (actionable staleness warning), and carries corpus sizes plus both split comparisons per the Epic: validated auto-loosening for auto-tune.ts's tuning advisor, gated on a clean backtest #8121 evidence-trail convention.
• Null — never a guess — on sample-floor misses or when nothing strictly dominates. Pure evaluation only: no cron, no alert, no writes, no new dependencies (per the issue's Boundaries).

Scope

• The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
• This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
• This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
• I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

• git diff --check
• npm run typecheck
• npm run actionlint
• npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
• npm run test:workers
• npm run build:mcp
• npm run test:mcp-pack
• npm run ui:openapi:check
• npm run ui:lint
• npm run ui:typecheck
• npm run ui:build
• npm audit --audit-level=moderate
• New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

• Ran the module's full suite (vitest run test/unit/loosening-knobs.test.ts — 14 tests green, including the 7 new drift cases mirroring the suite's membership-probe seeding technique per the issue's test requirement) plus the full root npm run typecheck and npm run build --workspace @loopover/engine. The new tests cover dominance in both directions, the shipped-value revert signal, the no-dominance null, the sample-floor null, held-out regression rejection, and byte-identical determinism — both sides of every conditional in the new evaluator. actionlint/workers/mcp/ui checks are untouched surfaces; CI runs them all.

Safety

• No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
• Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
• Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
• API/OpenAPI/MCP behavior is updated and tested where needed.
• UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
• Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
• Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

UI Evidence

Not applicable — pure backend evaluation function (no UI, docs, or extension surface touched; no cron, no writes, nothing visible changes until the separate wire issue consumes it).

Notes

• evaluateKnobLoosening itself is untouched — the drift evaluator sits beside it sharing the private compareOnSlice helper, so the two disciplines can never diverge on how a candidate is scored.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.55%. Comparing base (0c8d3d6) to head (e52fbf6).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8232      +/-   ##
==========================================
- Coverage   92.10%   89.55%   -2.55%     
==========================================
  Files         776       98     -678     
  Lines       78328    22651   -55677     
  Branches    23668     3876   -19792     
==========================================
- Hits        72147    20286   -51861     
+ Misses       5062     2187    -2875     
+ Partials     1119      178     -941     

Flag	Coverage Δ
shard-1	93.10% <100.00%> (+38.17%)	⬆️
shard-2	51.72% <0.00%> (-0.88%)	⬇️
shard-3	3.44% <0.00%> (-51.32%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/services/loosening-knobs.ts	100.00% <100.00%> (ø)

... and 678 files with indirect coverage changes

[loopover-orb]

Tip

✅ LoopOver review result - approve/merge recommended

_{Review updated: 2026-07-23 13:39:38 UTC}

2 files · 1 AI reviewer · no blockers · readiness 98/100 · CI green · clean

✅ Suggested Action - Approve/Merge

• safe to merge

Review summary
Adds evaluateKnobDrift, a pure sibling to evaluateKnobLoosening that checks whether the live value of a knob is still Pareto-optimal in either direction (not just whether it can be safely loosened). The candidate pool (shipped value + registry candidates, minus live, nearest-first with tighter-wins on ties) and split/floor discipline correctly mirror the existing loosening evaluator, and I traced the tie-break and 'shipped' direction logic against the test fixtures — both check out (e.g. tie_probe's 0.9 vs 0.85 equidistant case resolves to 0.9 as coded). This is pure evaluation code with no writes, no cron, no new deps, consistent with the issue's stated boundaries and closing #8212.

Nits — 3 non-blocking

• src/services/loosening-knobs.ts:160-181 — the JSDoc and inline comment on KnobDriftReport/evaluateKnobDrift are fairly verbose relative to the sibling evaluateKnobLoosening's docs; consider trimming to match house style once this pattern is used elsewhere.
• External brief flagged '8212' as an unexplained magic number at loosening-knobs.ts:160 — it's an issue-reference in a comment, not a literal used in logic, so no action needed.
• Consider whether evaluateKnobDrift's shared logic with evaluateKnobLoosening (split/floor check, compareOnSlice loop) could be factored into a small shared helper now that there are two near-identical scan loops, though this is optional given how deliberately the PR mirrors the original for auditability.

Decision drivers

• ✅ Code review — No blockers (1 reviewer)
• ✅ Gate result — Passing (No configured blocker found.)

Context & advisory signals — never blocks the verdict

Signal	Result	Evidence
Linked issue	✅ Linked	#8212
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 343 registered-repo PR(s), 136 merged, 36 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	RealDiligent; Gittensor profile; 343 PR(s), 36 issue(s).
Improvement	✅ Minor	risk: clean · value: minor · LLM: significant

Linked issue satisfaction

Addressed
The PR adds a pure evaluateKnobDrift function beside evaluateKnobLoosening that evaluates every registry candidate plus the shipped value, applies the same split/Pareto floor/sample-minimum discipline, distinguishes looser/tighter/shipped direction, and returns null on sample-floor misses or non-dominance, with corresponding tests covering both directions, no-dominance, sample-floor nulls, held-ou

Review context

• Author: RealDiligent
• Role context: outside_contributor
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: Python, Ruby, JavaScript, Svelte, TypeScript, Markdown, MDX, Rust
• Official Gittensor activity: 343 PR(s), 36 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Keep the PR focused and include validation evidence before maintainer review.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

• @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
• @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
• A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

• Re-run LoopOver review

[loopover-orb]

LoopOver approves — the gate is satisfied and CI is green.