Skip to content

fix(review): apply the prompt-injection reputation override before the minSample guard - #5975

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
jeffrey701:fix/5940-prompt-injection-before-minsample
Jul 15, 2026
Merged

fix(review): apply the prompt-injection reputation override before the minSample guard#5975
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
jeffrey701:fix/5940-prompt-injection-before-minsample

Conversation

@jeffrey701

Copy link
Copy Markdown
Contributor

Problem

signalFromCounts (src/review/submitter-reputation.ts) derives a submitter's reputation ("low" | "neutral" | "trusted") from recency-windowed outcome counts. Its own comment describes the prompt-injection check as an unconditional hard override — "the single hard-abuse signal", any one occurrence is enough. But the sample < cfg.minSample guard (default minSample: 5) runs first and returns "neutral" before the prompt-injection branch is ever reached:

const sample = c.success + c.qualityFail + c.qualityFailLight + c.promptInjection;
if (sample < cfg.minSample) return "neutral";      // ← returns first

// ── 'low' — genuine malice: ANY prompt-injection ──
if (c.promptInjection > 0) return "low";           // ← never reached for a low-sample account

So a first-time / low-history submitter whose only windowed outcome is a single source_prompt_injection close has sample = 1 < 5"neutral" — the exact opposite of the documented intent, and the worst case for this bug: a brand-new low-sample account attempting prompt injection once is precisely the scenario the hard override exists to catch.

Fix

Move the prompt-injection hard override above the minSample guard so it fires unconditionally, as documented. Nothing else changes — the weighted-fail / trusted logic still runs only over a sufficient sample.

const sample = c.success + c.qualityFail + c.qualityFailLight + c.promptInjection;

// ── 'low' — ANY prompt-injection (unconditional hard override; precedes the minSample guard) ──
if (c.promptInjection > 0) return "low";

if (sample < cfg.minSample) return "neutral";

Test

test/unit/submitter-reputation.test.ts adds: a single prompt-injection from a low-sample account (sample = 1) → "low" — which returned "neutral" before the fix. The existing sufficient-sample prompt-injection cases and the minSample/neutral cases are unchanged.

Local: the two affected unit suites pass; tsc --noEmit clean on the changed files.

Closes #5940

…e minSample guard

signalFromCounts returned "neutral" for a low-sample account whose only
windowed outcome was a single source_prompt_injection close, because the
sample < minSample guard ran before the documented unconditional
prompt-injection hard override. Move the override above the guard so any
prompt-injection returns "low" regardless of sample size, as the comment
already describes.

Closes JSONbored#5940
@jeffrey701
jeffrey701 requested a review from JSONbored as a code owner July 15, 2026 05:20
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.16%. Comparing base (4749ff7) to head (b349528).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5975   +/-   ##
=======================================
  Coverage   95.16%   95.16%           
=======================================
  Files         589      589           
  Lines       46790    46790           
  Branches    14943    14943           
=======================================
  Hits        44528    44528           
  Misses       1511     1511           
  Partials      751      751           
Flag Coverage Δ
shard-1 43.50% <0.00%> (-0.42%) ⬇️
shard-2 36.36% <0.00%> (+0.15%) ⬆️
shard-3 31.98% <100.00%> (ø)
shard-4 31.95% <0.00%> (ø)
shard-5 32.26% <0.00%> (-0.24%) ⬇️
shard-6 44.89% <100.00%> (+0.18%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/review/submitter-reputation.ts 100.00% <100.00%> (ø)

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 15, 2026
@loopover-orb

loopover-orb Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-15 05:27:02 UTC

2 files · 1 AI reviewer · no blockers · readiness 82/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This diff reorders the prompt-injection hard-override check above the minSample guard in signalFromCounts, matching the function's own documented intent that any prompt-injection is an unconditional hard-abuse signal. The fix is correct and minimal — it's a straight code-motion of one if-statement with no logic changes elsewhere, and the added test directly exercises the previously-broken path (sample=1 with a single injection now returns "low" instead of "neutral"). This closes issue #5940 as referenced in the test comment.

Nits — 2 non-blocking
  • src/review/submitter-reputation.ts: the comment block above the moved check is fairly long (4 lines) for a one-line override — could be tightened, though the added context is genuinely useful given the bug this fixes.
  • Consider also adding a test for sample=0 with promptInjection>0 to cover the absolute floor case, though sample=1 likely already exercises the same branch.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #5940
Related work ⚠️ 2 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 131 registered-repo PR(s), 63 merged, 33 issue(s).
Contributor context ✅ Confirmed Gittensor contributor jeffrey701; Gittensor profile; 131 PR(s), 33 issue(s).
Gate result ✅ Passing No configured blocker found.
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Addressed
The diff reorders signalFromCounts to check c.promptInjection > 0 before the minSample guard, exactly matching the issue's requested fix, and adds the required regression test for a low-sample (sample=1) prompt-injection case returning 'low', plus preserves existing minSample/neutral coverage for other branches.

Review context
Contributor next steps
  • Start here: Review top overlaps.
  • Then work through the remaining 2 steps in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoopOver approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit ac83483 into JSONbored:main Jul 15, 2026
15 checks passed
@nghetienhiep nghetienhiep mentioned this pull request Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(review): signalFromCounts's minSample guard masks the unconditional prompt-injection hard override

1 participant