Skip to content

fix(selfhost): stop claude-code from treating its own review call as an interactive plan or a prompt injection - #5080

Merged
JSONbored merged 1 commit into
mainfrom
fix/5079-claude-plan-mode-injection-lookalike
Jul 11, 2026
Merged

fix(selfhost): stop claude-code from treating its own review call as an interactive plan or a prompt injection#5080
JSONbored merged 1 commit into
mainfrom
fix/5079-claude-plan-mode-injection-lookalike

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Root-caused live via two real ai_review_provider_unparseable_output events minutes apart (visible for the first time thanks to #5071's diagnostic snippet), where the model explicitly refused to review, citing:

"Two things are going on that don't line up: 1. Plan Mode is active... 2. But the bulk of your message is a..."

"I want to flag something... this turn contains a very large embedded block claiming to be 'ADDITIONAL SYSTEM INSTRUCTIONS'... Layered on top of that is a 'Plan Mode' wr[apper]..."

Two compounding bugs in createClaudeCodeAi (src/selfhost/ai.ts):

  1. --permission-mode plan has been passed to every claude invocation since the self-host stack's original commit (2026-06-24, 0325fb1e). This activates the full interactive Plan-Mode workflow (explore → draft a plan → wait for ExitPlanMode approval), not just a read-only permission restriction — --disallowedTools Bash,Edit,Write,WebFetch,WebSearch already forbids every mutating/networked tool, so nothing needing that framing remains. Switched to bypassPermissions, matching the "no TTY to answer a permission prompt, --disallowedTools is the real boundary" intent this call already had.
  2. systemAppend (repo-level review instructions from .gittensory.yml) was textually prepended into the stdin prompt behind a literal "ADDITIONAL SYSTEM INSTRUCTIONS:\n" header. This was implemented correctly in feat(selfhost): wire claude --append-system-prompt for review instructions (v2 refinement) #1471 / PR feat(selfhost): pass repo instructions to cli reviewers #2954 via the real --append-system-prompt flag, then regressed by a later commit (85999110, "keep Claude review instructions out of argv", fix(selfhost): keep Claude review instructions out of argv #3951) that moved it back to textual stdin-smuggling to avoid the content appearing in ps aux — inadvertently recreating the textbook shape of a prompt-injection attack (a labeled "system instructions" block inside otherwise-untrusted content), which claude-code's own safety training correctly flags as suspicious. Now written to a file inside the same per-call isolated temp dir already used for the subprocess's cwd, passed via --append-system-prompt-fileresolves fix(selfhost): keep Claude review instructions out of argv #3951's original argv-visibility concern via the CLI's own file-based mechanism instead of textual smuggling.

codex is unaffected: it uses --sandbox read-only, not --permission-mode, and has no CLI-native system-prompt-file equivalent (per #1471's own documented limitation) — keeps its existing textual-prepend approach unchanged.

This has likely been silently degrading claude-code review quality for ~3 weeks.

Closes #5079

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (Closes #5079).

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage (full unsharded npm run test:ci, all green) — the only uncovered line in ai.ts (a buildProvider switch default) predates this diff.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries — rewrote the stale test that asserted the old (buggy) textual-smuggling behavior, added a dedicated --permission-mode bypassPermissions regression test, and verify the written system-prompt-file's on-disk content directly.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (N/A — no auth surface touched.)
  • API/OpenAPI/MCP behavior is updated and tested where needed. (N/A.)
  • UI changes use live API data or real empty/error/loading states. (N/A — no UI changed.)
  • Visible UI changes include a UI Evidence section. (N/A — backend-only.)
  • Public docs/changelogs are updated where needed. (N/A.)

Notes

bypassPermissions is safe here specifically because --disallowedTools already forbids every tool that would otherwise need an interactive approval; nothing new is granted.

…an interactive plan or a prompt injection (#5079)

Two compounding bugs in createClaudeCodeAi, confirmed live via two real
ai_review_provider_unparseable_output events where the model explicitly
refused, citing both:

1. --permission-mode plan has been passed to every claude invocation
   since the self-host stack's original commit. This activates the
   full interactive Plan-Mode workflow (explore, draft a plan, wait for
   ExitPlanMode approval), not just a read-only restriction --
   disallowedTools already forbids every mutating/networked tool, so
   nothing needing that framing remains. Switched to bypassPermissions,
   matching the "no TTY to answer a prompt, disallowedTools is the real
   boundary" intent this call already has.

2. systemAppend (repo-level review instructions from .gittensory.yml)
   was textually prepended into the stdin prompt behind a literal
   "ADDITIONAL SYSTEM INSTRUCTIONS:" header. This was implemented
   correctly in #1471/PR #2954 via the real --append-system-prompt
   flag, then regressed by a later commit that moved it back to
   textual stdin-smuggling to keep the content out of argv/`ps aux` --
   inadvertently recreating the textbook shape of a prompt-injection
   attack, which claude-code's own safety training correctly flags as
   suspicious. Now written to a file inside the same per-call isolated
   temp dir already used for the subprocess cwd, passed via
   --append-system-prompt-file: content stays out of argv AND out of
   the untrusted-looking stdin blob.

codex is unaffected (--sandbox read-only, not --permission-mode; no
CLI-native system-prompt-file equivalent, per #1471's own documented
limitation -- keeps its existing textual-prepend approach).
@JSONbored JSONbored self-assigned this Jul 11, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.19%. Comparing base (99c2429) to head (bf02cb7).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5080   +/-   ##
=======================================
  Coverage   94.18%   94.19%           
=======================================
  Files         468      468           
  Lines       39616    39623    +7     
  Branches    14456    14457    +1     
=======================================
+ Hits        37314    37321    +7     
  Misses       1646     1646           
  Partials      656      656           
Flag Coverage Δ
shard-1 46.46% <0.00%> (-0.02%) ⬇️
shard-2 33.62% <100.00%> (-0.25%) ⬇️
shard-3 31.38% <0.00%> (-0.82%) ⬇️
shard-4 32.68% <0.00%> (+0.72%) ⬆️
shard-5 33.20% <0.00%> (-0.10%) ⬇️
shard-6 45.16% <0.00%> (+0.20%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/selfhost/ai.ts 98.08% <100.00%> (+0.02%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored
JSONbored merged commit 6784974 into main Jul 11, 2026
18 checks passed
@JSONbored
JSONbored deleted the fix/5079-claude-plan-mode-injection-lookalike branch July 11, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

claude-code reviewer runs in interactive Plan Mode and treats its own review instructions as a prompt injection

1 participant