Skip to content

fix(review): signalFromCounts's minSample guard masks the unconditional prompt-injection hard override #5940

Description

@JSONbored

Context

signalFromCounts in src/review/submitter-reputation.ts (lines 208-224) derives a submitter's reputation signal ("low" | "neutral" | "trusted") from recency-windowed outcome counts:

export function signalFromCounts(c: ReputationCounts, cfg: ReputationConfig = DEFAULT_REPUTATION_CONFIG): ReputationSignal {
  const weightedFails = c.qualityFail + c.qualityFailLight * cfg.lightFailWeight;
  const sample = c.success + c.qualityFail + c.qualityFailLight + c.promptInjection;
  if (sample < cfg.minSample) return "neutral";

  // ── 'low' — genuine malice: ANY prompt-injection (the single hard-abuse signal). ──
  if (c.promptInjection > 0) return "low";
  ...

The function's own comment describes the prompt-injection check as an unconditional hard override — "the single hard-abuse signal", any one occurrence is enough. But the sample < cfg.minSample guard (default minSample: 5) runs first and returns "neutral" before the prompt-injection check is ever reached.

Concretely: a first-time or low-history submitter whose only windowed outcome is a single source_prompt_injection close has sample = 1 < 5, so signalFromCounts returns "neutral" — the exact opposite of the documented intent, and the worst case for this bug, since a low-history account attempting prompt injection is precisely the scenario the hard override exists to catch (a would-be repeat abuser doesn't need the override as urgently; a brand-new low-sample account attempting it once is where the override matters most).

test/unit/submitter-reputation.test.ts only exercises the prompt-injection branch with sample padded to >= minSample (e.g. mixed with enough success/qualityFail rows to clear the threshold), so the low-sample bypass is untested and unnoticed.

Requirements

  • Reorder signalFromCounts so the c.promptInjection > 0 → "low" check runs before the sample < cfg.minSample → "neutral" guard, matching the function's own documented "any one is enough" intent for prompt injection.
  • Do not change the minSample gate's effect on the other two branches (qualityFailLowRate/trustedMinSuccess) — those should still return "neutral" for sample < cfg.minSample exactly as today.
  • Do not touch DEFAULT_REPUTATION_CONFIG values or ReputationConfig/ReputationCounts types — this is a pure control-flow ordering fix inside signalFromCounts.
  • Do not touch anything outside signalFromCounts (its caller in src/review/submitter-reputation.ts and downstream trust-tier consumers are out of scope for this fix).

Deliverables

  • signalFromCounts returns "low" whenever c.promptInjection > 0, regardless of sample size.
  • A regression test in test/unit/submitter-reputation.test.ts asserting signalFromCounts({ success: 0, qualityFail: 0, qualityFailLight: 0, promptInjection: 1 }, DEFAULT_REPUTATION_CONFIG) returns "low" (a low-sample account with a single prompt-injection close).
  • A test confirming the pre-existing sample < minSample → "neutral" behavior for the other branches (no prompt injection present) is unchanged.

Test Coverage Requirements

Aim for 99%+ Codecov patch coverage (100% including the new branch/invariant) on the touched lines in src/review/submitter-reputation.ts. This is a fix for a real reputation-signal precedence bug, so the low-sample-with-prompt-injection regression test is required, not just incidental line coverage.

Expected Outcome

A submitter with any windowed prompt-injection close is scored "low" reputation regardless of overall sample size, matching the module's own documented hard-override intent instead of being masked by the minSample guard.

Links & Resources

  • src/review/submitter-reputation.ts (signalFromCounts, lines 208-224; DEFAULT_REPUTATION_CONFIG, lines 195-203)
  • test/unit/submitter-reputation.test.ts (existing test suite)

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions