fix(selfhost): stop claude-code from treating its own review call as an interactive plan or a prompt injection - #5080
Merged
Conversation
…an interactive plan or a prompt injection (#5079) Two compounding bugs in createClaudeCodeAi, confirmed live via two real ai_review_provider_unparseable_output events where the model explicitly refused, citing both: 1. --permission-mode plan has been passed to every claude invocation since the self-host stack's original commit. This activates the full interactive Plan-Mode workflow (explore, draft a plan, wait for ExitPlanMode approval), not just a read-only restriction -- disallowedTools already forbids every mutating/networked tool, so nothing needing that framing remains. Switched to bypassPermissions, matching the "no TTY to answer a prompt, disallowedTools is the real boundary" intent this call already has. 2. systemAppend (repo-level review instructions from .gittensory.yml) was textually prepended into the stdin prompt behind a literal "ADDITIONAL SYSTEM INSTRUCTIONS:" header. This was implemented correctly in #1471/PR #2954 via the real --append-system-prompt flag, then regressed by a later commit that moved it back to textual stdin-smuggling to keep the content out of argv/`ps aux` -- inadvertently recreating the textbook shape of a prompt-injection attack, which claude-code's own safety training correctly flags as suspicious. Now written to a file inside the same per-call isolated temp dir already used for the subprocess cwd, passed via --append-system-prompt-file: content stays out of argv AND out of the untrusted-looking stdin blob. codex is unaffected (--sandbox read-only, not --permission-mode; no CLI-native system-prompt-file equivalent, per #1471's own documented limitation -- keeps its existing textual-prepend approach).
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5080 +/- ##
=======================================
Coverage 94.18% 94.19%
=======================================
Files 468 468
Lines 39616 39623 +7
Branches 14456 14457 +1
=======================================
+ Hits 37314 37321 +7
Misses 1646 1646
Partials 656 656
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Root-caused live via two real
ai_review_provider_unparseable_outputevents minutes apart (visible for the first time thanks to #5071's diagnostic snippet), where the model explicitly refused to review, citing:Two compounding bugs in
createClaudeCodeAi(src/selfhost/ai.ts):--permission-mode planhas been passed to everyclaudeinvocation since the self-host stack's original commit (2026-06-24,0325fb1e). This activates the full interactive Plan-Mode workflow (explore → draft a plan → wait forExitPlanModeapproval), not just a read-only permission restriction —--disallowedTools Bash,Edit,Write,WebFetch,WebSearchalready forbids every mutating/networked tool, so nothing needing that framing remains. Switched tobypassPermissions, matching the "no TTY to answer a permission prompt,--disallowedToolsis the real boundary" intent this call already had.systemAppend(repo-level review instructions from.gittensory.yml) was textually prepended into the stdin prompt behind a literal"ADDITIONAL SYSTEM INSTRUCTIONS:\n"header. This was implemented correctly in feat(selfhost): wire claude --append-system-prompt for review instructions (v2 refinement) #1471 / PR feat(selfhost): pass repo instructions to cli reviewers #2954 via the real--append-system-promptflag, then regressed by a later commit (85999110, "keep Claude review instructions out of argv", fix(selfhost): keep Claude review instructions out of argv #3951) that moved it back to textual stdin-smuggling to avoid the content appearing inps aux— inadvertently recreating the textbook shape of a prompt-injection attack (a labeled "system instructions" block inside otherwise-untrusted content), which claude-code's own safety training correctly flags as suspicious. Now written to a file inside the same per-call isolated temp dir already used for the subprocess'scwd, passed via--append-system-prompt-file— resolves fix(selfhost): keep Claude review instructions out of argv #3951's original argv-visibility concern via the CLI's own file-based mechanism instead of textual smuggling.codexis unaffected: it uses--sandbox read-only, not--permission-mode, and has no CLI-native system-prompt-file equivalent (per #1471's own documented limitation) — keeps its existing textual-prepend approach unchanged.This has likely been silently degrading
claude-codereview quality for ~3 weeks.Closes #5079
Scope
type(scope): short summaryConventional Commit format, for examplefix(api): restore profile access checks.CONTRIBUTING.mdand does not reintroduce GitHub Pages, VitePress,site/, orCNAME.Closes #5079).Validation
git diff --checknpm run actionlintnpm run typechecknpm run test:coverage(full unshardednpm run test:ci, all green) — the only uncovered line inai.ts(abuildProviderswitch default) predates this diff.npm run test:workersnpm run build:mcpnpm run test:mcp-packnpm run ui:openapi:checknpm run ui:lintnpm run ui:typechecknpm run ui:buildnpm audit --audit-level=moderate--permission-mode bypassPermissionsregression test, and verify the written system-prompt-file's on-disk content directly.Safety
UI Evidencesection. (N/A — backend-only.)Notes
bypassPermissionsis safe here specifically because--disallowedToolsalready forbids every tool that would otherwise need an interactive approval; nothing new is granted.