Skip to content

orb(security): AI reviewer prompt has no instruction-hierarchy defense — untrusted PR text is concatenated raw #9035

Description

@JSONbored

The AI reviewer's verdict can auto-merge or block a PR, and attacker-controlled text reaches its prompt with no structural defense — only regex defang.

  • Title, body, and raw diff are concatenated straight into the user prompt with no delimiting or instruction-hierarchy markers: src/services/ai-review.ts ~1019-1037 (Pull request #${n}: ${title}, Description:\n${body}, diff.slice(0,120000)). The system prompt (REVIEW_SYSTEM_PROMPT ~77-102) says "judge ONLY the diff" but never fences the untrusted regions.
  • The sole defense is regex defang (src/review/prompt-injection.ts ~23-62) — deliberately narrow, bypassable by paraphrase/encoding/structure — and by design it NEVER changes the verdict (~86-90).
  • The verdict is whatever JSON the model emits, trusted structurally by parseModelReview (~830-949). Because the same injected text reaches BOTH consensus reviewers, a successful injection suppresses both and avoids even the ai_review_split single-rejection rule (ai-review-orchestration.ts ~770-794).

Good news, verified live on edge-nl-01: LOOPOVER_REVIEW_SAFETY=true, so defang is active on this deployment — but it remains per-repo feature-gated (convergedFeatureActive(..., "safety"), ai-review.ts ~2424-2431) and is regex-only. Default for self-host operators is OFF (src/review/safety.ts ~24-29).

Fix

  1. Wrap every untrusted field in explicit delimiters with a system rule that content between markers is DATA, never instructions.
  2. Enable defang unconditionally in block mode rather than behind a flag.
  3. Treat a detected injection attempt as a signal that forces a manual hold — currently it is recorded but deliberately never affects the verdict, which means a caught attacker still gets a normal roll.

Acceptance

  • A PR body containing reviewer-directed instructions is fenced as data and additionally routed to a human hold.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.orbGittensory Orb related - maintainer self-hosting analytics.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions