feat(scoring): act on unmodeled-constant drift (Fixes #811)

[kiannidev]

Summary

• Route findUnmodeledUpstreamConstants output into the existing Pipeline-B upstream drift surface via a stable-fingerprint report (scoring_model / unmodeled_scoring_constants).
• Sync on both refreshScoringModelSnapshot and refreshUpstreamDrift, resolving the report when all upstream constants are modeled.
• Existing fileUpstreamDriftIssues + GITTENSORY_AUTO_FILE_DRIFT_ISSUES can now auto-file GitHub issues for this signal.

Test plan

• npx vitest run test/unit/unmodeled-scoring-drift.test.ts
• npx vitest run test/unit/scoring.test.ts test/unit/upstream-ruleset.test.ts
• With GITTENSORY_AUTO_FILE_DRIFT_ISSUES=true, confirm scheduled file-upstream-drift-issues creates/updates an issue when upstream adds an unmodeled constant

Fixes #811

[ghost]

Note

Gittensory Gate skipped

PR closed before full evaluation. No late first comment was created.

Signal	Result	Evidence	Action
Gate result	⚠️ Skipped	#832 is no longer open.	No action.

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[ghost]

gittensory · advisory review

Reviewed 6 changed file(s) — two independent AI reviewers.

Changed files: src/scoring/model.ts, src/upstream/ruleset.ts, src/upstream/unmodeled-scoring-drift.ts, test/unit/scoring.test.ts, test/unit/unmodeled-scoring-drift.test.ts, test/unit/upstream-ruleset.test.ts

Suggested action: ✅ Safe to merge — both reviewers found no blocking issues.

Reviewer A · gpt-oss-120b — recommends ✅ merge
The PR introduces drift detection for unmodeled scoring constants, adds a new utility module, integrates it into scoring model refresh and upstream drift refresh, and provides comprehensive tests. The implementation follows existing patterns and respects the public/private sanitization rules.

Suggestions

• Document the new findUnmodeledConstantKeys export in the scoring model module README or JSDoc for future maintainers.
• Consider adding a unit test that simulates a DB failure during upsertUpstreamDriftReport to verify graceful error handling.
• Verify that the Node test environment includes crypto.randomUUID; if not, polyfill or import from the crypto module.

Worth double-checking

• Performance impact of scanning up to 50 drift reports on each refresh; monitor if this becomes a bottleneck.
• Potential future changes to the fingerprint seed must be coordinated to avoid breaking existing reports.

Reviewer B · nemotron-3-120b-a12b — recommends ✅ merge
The PR adds detection and reporting of unmodeled scoring constants from upstream (Gittensor) to improve visibility of drift. It introduces a new drift report type, integrates it into scoring model and ruleset refresh flows, and includes comprehensive unit tests. The changes are cohesive, well-tested, and maintain the public/private boundary (no forbidden terms in drift reports).

Suggestions

• Consider adding a comment in src/upstream/unmodeled-scoring-drift.ts explaining why SCORING_MODEL_FOLLOW_UP includes specific files (to aid future maintenance).
• In test/unit/unmodeled-scoring-drift.test.ts, the test 'preserves linked issue metadata' could be strengthened by verifying that the issue metadata is preserved when the report is resolved (though current behavior is acceptable).
• The hardcoded upstream repo/ref fallbacks ('entrius/gittensor', 'test') should be verified against documentation to ensure they match expected defaults.

Worth double-checking

• Duplicate drift report generation: syncUnmodeledScoringConstantDrift is called from both refreshScoringModelSnapshot and refreshUpstreamDrift. While deduplicated by fingerprint, ensure this doesn't cause unnecessary writes during frequent runs.
• The severity threshold (>=3 constants = high) is reasonable but should be revisited if false positives arise; consider making it configurable via env in future.
• Ensure the drift report's 'recommendedFollowUp' array stays accurate if file paths change (though this is low-risk as it's advisory).

[codecov]

Codecov Report

❌ Patch coverage is 97.14286% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 96.71%. Comparing base (e879ffa) to head (31c84fb).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/upstream/ruleset.ts	75.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #832   +/-   ##
=======================================
  Coverage   96.71%   96.71%           
=======================================
  Files         108      109    +1     
  Lines       14641    14674   +33     
  Branches     5298     5314   +16     
=======================================
+ Hits        14160    14192   +32     
  Misses        102      102           
- Partials      379      380    +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[superagent-security]

⚠️ Contributor trust inconclusive. Click here for more info: Superagent Dashboard

[JSONbored]

Good:

• Cleanly adds unmodeled upstream scoring-constant drift detection.
• Integrates with scoring refresh and upstream drift refresh.
• Tests cover create/resolve/severity behavior.
• CI, Codecov, Superagent, and Gittensory gate are passing.

Bad:

• Minor metadata nit: scoring-model refresh hardcodes entrius/gittensor:test in one source path.

Change requests:

• Fix the metadata nit.

[kiannidev]

@JSONbored
I addressed feedback.
Please review PR again.

[JSONbored]

PR #832 — feat(scoring): act on unmodeled-constant drift (Fixes #811) [kiannidev]
Action: APPROVE
Issue #811: FULLY CI: pass
Good:

• syncUnmodeledScoringConstantDrift opens/updates/resolves a stable-fingerprint upstream drift report
  from both refreshScoringModelSnapshot and refreshUpstreamDrift; severity scales with count.
• The maintainer's only nit (hardcoded entrius/gittensor:test) is resolved — source is now env-driven
  (GITTENSOR_UPSTREAM_REPO/REF) with a fallback, and a test asserts the override.
• 9-case unit suite + integration coverage. On-direction backend work.