Repository navigation
test(e2e): verify Claude is prefilled with issue URL on start - #9980
Conversation
Regression test for #6613: when starting a workspace from a newly created GitHub issue, ensure the issue URL is passed to Claude via `--prefill` and `--dangerously-skip-permissions` flags. This prevents context loss after issue creation.
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds a Playwright end-to-end test with fake platform-specific 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
tests/e2e/github-created-issue-start-prefill.spec.ts (2)
164-164: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUnnecessary
RegExpfor substring matching.Playwright's
getByRolenameoption already does case-insensitive substring matching for plain strings by default; wrappingISSUE_TITLEinRegExpis redundant and is what triggers the static-analysis ReDoS warning. Passing the string directly avoids the warning without changing matching behavior.- await expect(orcaPage.getByRole('heading', { name: new RegExp(ISSUE_TITLE) })).toBeVisible({ + await expect(orcaPage.getByRole('heading', { name: ISSUE_TITLE })).toBeVisible({Source: Linters/SAST tools
170-180: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRedundant terminal read after the poll succeeds.
The polled content at line 171 is discarded, then
getTerminalContentis called again at line 176 for the remaining assertions — an extra IPC round-trip and a small window where content could theoretically differ between the two reads. Capture the polled value directly instead.♻️ Proposed fix
- await expect - .poll(() => getTerminalContent(orcaPage, 12_000), { - timeout: 30_000, - message: 'Claude prefill command did not reach the active terminal buffer' - }) - .toContain('--prefill') - const terminalText = await getTerminalContent(orcaPage, 12_000) + let terminalText = '' + await expect + .poll( + async () => { + terminalText = await getTerminalContent(orcaPage, 12_000) + return terminalText + }, + { + timeout: 30_000, + message: 'Claude prefill command did not reach the active terminal buffer' + } + ) + .toContain('--prefill')
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: c3cb8b53-d533-41b3-a7ca-8b0cba271eb6
📒 Files selected for processing (1)
tests/e2e/github-created-issue-start-prefill.spec.ts
- Reorder mock API handlers to ensure `/labels` and `/assignees` paths match before the specific issue endpoint - Replace regex heading matcher with exact string for more reliable assertions - Refactor terminal content polling to capture text once and reuse in subsequent assertions
…ai#9980) * test(e2e): verify Claude is prefilled with issue URL on start Regression test for stablyai#6613: when starting a workspace from a newly created GitHub issue, ensure the issue URL is passed to Claude via `--prefill` and `--dangerously-skip-permissions` flags. This prevents context loss after issue creation. * test(e2e): fix GitHub-issue-start prefill test flakiness - Reorder mock API handlers to ensure `/labels` and `/assignees` paths match before the specific issue endpoint - Replace regex heading matcher with exact string for more reliable assertions - Refactor terminal content polling to capture text once and reuse in subsequent assertions
Summary
Adds E2E test to verify that when a user creates a new GitHub issue in Orca and starts a workspace from it, the Claude agent is launched with the issue URL prefilled via the
--prefillflag.This test validates the fix for the regression where newly created issues were not properly passing their context (URL) to Claude on start.
Testing
--prefilland--dangerously-skip-permissionsflags with the issue URLTest uses mocked
ghandclaudeCLIs to simulate GitHub interaction and capture command arguments. Cross-platform paths handled for Windows and Unix-like systems.Security Audit
No security concerns. Test file only; uses mocked CLIs and a disposable E2E repository. Fake CLI scripts are isolated to a temporary directory and cleaned up after the test suite.
Notes
Test covers:
.cmdbatch scripts on Windows, executable shell scripts on Unix)launchEnvto inject fake CLI directory into PATH