feat(scoring): surface upstream scoring constants gittensory does not model (staleness visibility) - #690
Conversation
… model (staleness visibility)
Makes scoring drift VISIBLE — the safe half of the scoring-currency work (the
ranking-changing time-decay implementation is deferred for owner review with a
before/after diff, per decision).
The upstream constants parse is `knownOnly` (it keeps only constants we already
encode), which silently HID upstream ADDITIONS — so when gittensor introduced new
scoring dimensions (e.g. the TIME_DECAY_* constants) gittensory drifted behind
with no signal. New findUnmodeledUpstreamConstants() detects constants upstream
defines that we don't model; refreshScoringModelSnapshot now records them on the
snapshot payload (unmodeledUpstreamConstants) AND raises a warning
("Upstream gittensor defines N scoring constant(s) gittensory does not yet model:
…. Scoring may be behind upstream."), which surfaces on /v1/scoring/model,
/v1/upstream/status, and the operator dashboard. Detection only — changes no score.
(The live upstream-contract test already runs on a schedule via the existing
upstream-contract.yml workflow, so that half was already covered.)
Tests: findUnmodeledUpstreamConstants flags TIME_DECAY_* (not modeled) and not
SRC_TOK_SATURATION_SCALE (modeled); refresh raises the staleness warning + records
the payload field. 97% branch.
Part of #525.
|
Note Gittensory Gate skippedPR closed before full evaluation. No late first comment was created.
💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers. |
ghost
added
the
|
gittensory · advisory review Reviewed 2 changed file(s) — two independent AI reviewers. Suggested action: 🛠️ Request changes. (reviewers split: request changes / merge) Address the suggestions below before merging. Reviewer A · Suggestions
Worth double-checking
Reviewer B · No blocking issues spotted. |
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
…f flag (#703) (#731) Ports upstream gittensor's sigmoid time-decay (the validator's calculate_time_decay) into gittensory's score model. DEFAULT-OFF + owner-gated: the roadmap defers #703 pending owner review of a before/after ranking diff, so this changes nothing live until SCORING_TIME_DECAY_ENABLED is set. - scoring/model.ts: model the 4 upstream constants (TIME_DECAY_GRACE_PERIOD_HOURS 12, SIGMOID_MIDPOINT 10, SIGMOID_STEEPNESS_SCALAR 0.4, MIN_MULTIPLIER 0.05) so they no longer surface as unmodeled drift (#690); + isTimeDecayEnabled gate (default off). - scoring/preview.ts: calculateTimeDecay (verbatim port: hard 12h grace -> 1.0, then a logistic on days-since-merge, 50% at 10d, floored at 0.05) + a timeDecayMultiplier applied to estimatedMergedScore. A fresh PR (age < grace) is unaffected even when enabled, so a normal new-PR preview never changes. - prAgeHours input + the owner-gated applyTimeDecay flag injected server-side at the routes + MCP score-preview entry points (not caller-controllable). Why per-PR projection (not a historical-aggregate refactor): gittensory's preview scores a single fresh planned PR and surfaces the contributor's upstream-API standing (already decayed upstream); it has no raw historical per-PR aggregation to decay, so re-applying there would double-count. The faithful model is per-PR (predicted score x decay-by-age). Decay curve (the before/after to review): fresh 100% -> 5d ~88% -> 10d 50% -> >=20d 5% floor. Tests: calculateTimeDecay vs upstream (grace, 50%@10D, floor, monotonic), constants now modeled, the env gate, the preview applied/unapplied paths, and a before/after trajectory. 97% coverage held; OpenAPI regenerated.
1 participant
The safe half of the scoring-currency work — makes drift visible without changing any score. (The ranking-changing time-decay implementation is deferred for your review with a before/after ranking diff, per your decision.)
The gap
The upstream
constants.pyparse isknownOnly— it keeps only constants gittensory already encodes, which silently hid upstream ADDITIONS. So when gittensor introduced new scoring dimensions (e.g. theTIME_DECAY_*constants the audit found), gittensory drifted behind with no signal at all.Fix (detection only — no score changes)
findUnmodeledUpstreamConstants()detects numeric constants upstream defines that gittensory doesn't model.refreshScoringModelSnapshotnow records them on the snapshot payload (unmodeledUpstreamConstants) and raises a warning — "Upstream gittensor defines N scoring constant(s) gittensory does not yet model: … . Scoring may be behind upstream." — which surfaces on/v1/scoring/model,/v1/upstream/status, and the operator dashboard. So "are we out of date?" is now answered automatically.The live upstream-contract test already runs on a schedule via the existing
upstream-contract.ymlworkflow (the audit was wrong that it's CI-skipped), so that half was already covered.Tests
findUnmodeledUpstreamConstantsflagsTIME_DECAY_*(unmodeled) and notSRC_TOK_SATURATION_SCALE(modeled); refresh raises the staleness warning + records the payload field.Verification
typecheck✅ ·test:coverage✅ (97.01% branch, 1696 tests) ·test:workers✅ ·git diff --check✅Part of #525. Follow-up (your review): implement the
TIME_DECAY_*scoring dimension to match upstream — changes live rankings, so I'll bring a before/after diff before merging.