test(review): add adversarial prompt-injection regression fixtures - #3565
Conversation
Expand test/unit/prompt-injection.test.ts with realistic reviewer- manipulation techniques (line-break splitting, HTML-comment hiding, markdown emphasis, all-caps shouting, fake role labels, diff-comment hiding, commit-trailer framing, zero-width-space insertion, roleplay framing) plus a documented-limitation section pinning what the current pattern list deliberately does not catch (Unicode homoglyphs, open vocabulary, over-budget gaps) and a false-positive section over realistic benign PR content. Fix a real gap the fixtures found: the injection regex's bounded gaps excluded newlines entirely, so "ignore all previous\ninstructions" (a single line break) slipped past detection even though the character budget was still respected. Removing the newline exclusion (keeping the period-boundary and character-count bounds, which is what actually prevents over-broad matches) closes it with no ReDoS risk (all gaps stay fixed-upper-bounded) and no new false positives. Also adds one end-to-end wiring test through the real runGittensoryAiReview pipeline for the specific "hidden in a diff code comment" scenario, confirming the manipulation span is redacted while the surrounding legitimate diff content is untouched. Closes #2998
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
|
Tip 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 ✅ Gittensory review result - approve/merge recommendedReview updated: 2026-07-05 17:22:47 UTC
✅ Suggested Action - Approve/Merge
Review summary Nits — 6 non-blocking
Review context
Contributor next steps
Signal definitions
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3565 +/- ##
=======================================
Coverage 93.11% 93.11%
=======================================
Files 304 304
Lines 31530 31530
Branches 11508 11508
=======================================
Hits 29359 29359
Misses 1517 1517
Partials 654 654
🚀 New features to boost your workflow:
|
Closes #2998
Summary
src/services/ai-review.tssends untrusted PR title/body/diff/changed-file-paths into the AI reviewer prompt (defanged only when the opt-infeatures.safetyflag is on, persrc/review/safety.ts). The existingtest/unit/prompt-injection.test.tshad exactly one example per detection category — no adversarial variations, no documented limitations, no diff-realistic fixture..gitignorechange, rubric documentation).[^.\n]{0,40}etc.) excluded newlines entirely, so"ignore all previous\ninstructions"— a single line break, well within the character budget — slipped past detection. Removed the newline exclusion insrc/review/prompt-injection.ts(keeping the period-boundary and character-count bounds, which are what actually prevent an over-broad match spanning two unrelated sentences). No ReDoS risk: every gap stays a fixed, small, bounded quantifier ({0,24}–{0,40}), never unbounded.runGittensoryAiReviewpipeline (test/unit/safety-wiring.test.ts), using the existingcapturingAiEnvhelper, for the issue's specific "hidden instruction embedded in a diff comment" scenario — confirms the manipulation span is redacted while the surrounding legitimate diff content reaches the model untouched.GittensoryAiReviewInput.diffis typed as "a bounded unified-diff-ish string built by the caller (filenames + patches)" — commit messages are a separate GitHub API concept (GET /commits) that this codebase never fetches into the AI review input. This attack vector has no path to the model today; noted as a comment in the test file rather than exercised as a redundant test.Scope
type(scope): short summaryConventional Commit format, for examplefix(api): restore profile access checks.CONTRIBUTING.mdand does not reintroduce GitHub Pages, VitePress,site/, orCNAME.Validation
git diff --checknpm run typecheck(clean)npx vitest run test/unit/prompt-injection.test.ts test/unit/safety-wiring.test.ts test/unit/safety.test.ts test/unit/ai-review.test.ts— 178/178 passingvitest --coverage --coverage.include=src/review/prompt-injection.ts): 100% statements/branches/functions/lines on the one touched source file.npm run test:workers/npm run build:mcp/npm run test:mcp-pack/npm run ui:openapi:check/npm run ui:build— not run individually; no worker/MCP/OpenAPI/UI surface touched.Safety
UI Evidencesection below. — N/A, no visible UI change.