Skip to content

feat(reporters): add onTestCaseAttempt hook - #11429

Closed
hi-ogawa wants to merge 1 commit into
attempts-api-prototypefrom
attempt-reporter-hook
Closed

hi-ogawa wants to merge 1 commit into
attempts-api-prototypefrom
attempt-reporter-hook

Conversation

@hi-ogawa

@hi-ogawa hi-ogawa commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

TODO: review

TestCase.attempts() from #11232 fills in while a test runs, but no reporter hook fires when a single run finishes, and the existing test-retried event never reaches a reporter hook. This PR adds a test-attempt-finished runner event that is sent after every retry and repeat run. It carries that run's attempt and calls a new onTestCaseAttempt(testCase, attempt) reporter hook. test-retried is kept but marked deprecated.

onTestCaseReady    |flaky|
  onHookStart      |flaky| [beforeEach]
  onHookEnd        |flaky| [beforeEach]
onTestCaseAttempt  |flaky| [repeat 0, retry 0, failed]
  onHookStart      |flaky| [beforeEach]
  onHookEnd        |flaky| [beforeEach]
onTestCaseAttempt  |flaky| [repeat 0, retry 1, passed]
  ...
onTestCaseAttempt  |flaky| [repeat 1, retry 1, passed]
onTestCaseResult   |flaky|

Because the event carries its own attempt, the hook always gets the right run even though task updates are batched. Tests configured with retry or repeats also send the batch immediately after each attempt, so testCase.attempts() inside the hook ends with that attempt. Tests that run only once keep the normal 100ms batching, so they don't pay for an extra message.

path how attempts are delivered
node pools test-attempt-finished event with { attempt } data
browser mode same event, forwarded unchanged by the existing onTaskUpdate RPC
--merge-reports convertTasksToEvents replays one event per saved attempt before test-finished

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hi-ogawa

hi-ogawa commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator Author

This appears less immediate need than #11232 so closing. Feel free to take over if any interest or create a new issue request if there's a need.

@hi-ogawa hi-ogawa closed this Oct 6, 2026
@hi-ogawa
hi-ogawa deleted the attempt-reporter-hook branch October 6, 2026 00:06
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant