feat: expose per-retry/repeat attempt result by hi-ogawa · Pull Request #11232 · vitest-dev/vitest

hi-ogawa · 2026-09-11T03:02:17Z

Description

This adds an TestCase.attempts() API so reporters can inspect the normalized result, errors, timing, retry count, and repeat index of each test attempt.

TODO

doc
follow-up?: add an onTestCaseAttempt(testCase, attempt) reporter hook.
- Reporters can't currently react to each attempt as it finishes.
- Send one test-attempt-finished event per attempt, covering both retries and repeats. Deprecate the existing test-retried.
- feat(reporters): add onTestCaseAttempt hook #11429

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

# Conflicts: # packages/vitest/src/runtime/runner/run.ts # test/e2e/test/repeats.test.ts # test/e2e/test/retry.test.ts # test/unit/test/repeats.test.ts

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Wait for vitest-dev#11232 to pick the trace of each error's attempt. This reverts commit dfc1db1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

sheremet-va · 2026-10-19T09:00:17Z

Conflict! 😢

# Conflicts: # packages/vitest/src/runtime/runner/run.ts

hi-ogawa and others added 30 commits September 9, 2026 17:11

fix(runner): apply fails before retrying

9c2f2ab

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): cover fails with retries and repeats

326c386

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): reproduce lost failure across repeats

2d62978

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): contrast ordinary and fails repeats

1851ea8

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): record current repeat results

7b05bbc

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): assert completed diagnostics exist

5f9d131

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

fix(runner): retain failures across repeats

7d9ec54

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): cover repeat failure aggregation

5716687

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): consolidate retry coverage

fdd91ab

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): update fails lifecycle expectations

829b6d3

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): clarify retry count across repeats

47f5304

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): show retry count across repeats

eb84349

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

docs(runner): clarify retry and repeat counts

4070bb6

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): assert retry result exists

129a889

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): reproduce lost repeat failures

d63d16c

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

fix(runner): retain failures across repeats

444d633

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): cover repeat failure hooks

4d9658d

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

chore: stack repeat results fix

15a8597

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

docs(runner): clarify retry and repeat counts

203953e

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

chore: update repeat results base

7e998d3

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): show retry counts across repeats

dedca80

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

chore: update repeat results base

a756764

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): alternate repeat outcomes

1b5bc45

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

chore: update repeat results base

11f3e68

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

test(runner): move repeat retries to e2e

88cf0a9

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

chore: update repeat results base

a9178d6

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

docs(runner): explain state widening

8073168

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

chore: update repeat results base

c39b768

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

docs(runner): clarify state widening

a32d934

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

chore: update repeat results base

2908873

Co-authored-by: Codex (GPT-6) <noreply@openai.com>

hi-ogawa added this to the 5.1.0 milestone Sep 29, 2026

Base automatically changed from fails-retry to main September 29, 2026 07:21

hi-ogawa and others added 5 commits October 1, 2026 18:38

Merge remote-tracking branch 'upstream/main' into attempts-api-prototype

becc514

# Conflicts: # packages/vitest/src/runtime/runner/run.ts # test/e2e/test/repeats.test.ts # test/e2e/test/retry.test.ts # test/unit/test/repeats.test.ts

refactor(reporters): share attempt state type

06f96ac

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

feat(runner): grow attempts while the test runs

68368d0

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

docs: document test attempts API

62f00aa

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(reporters): cover repeated test attempts

f372728

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

hi-ogawa mentioned this pull request Oct 1, 2026

feat(reporters): add onTestCaseAttempt hook #11429

Closed

hi-ogawa and others added 10 commits October 1, 2026 19:27

test(runner): make growing attempts test readable

86827cd

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(runner): name attempt fields in growing attempts test

330cb80

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(runner): name observation fields in growing attempts test

ccad3c2

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(runner): observe growing attempts from afterEach

d9837ba

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(runner): cover skipped attempts

17a9892

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(runner): note afterEach sees previous attempts only

8651715

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(runner): use attempt field names as is

effbcfb

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(runner): diagram attempts in growing attempts test

2bd2784

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(runner): use minimal retry in growing attempts test

318a66e

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(runner): diagram skipped attempts test

94fbd47

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

hi-ogawa marked this pull request as ready for review October 1, 2026 10:42

sheremet-va approved these changes Oct 5, 2026

View reviewed changes

hi-ogawa mentioned this pull request Oct 16, 2026

Failure screenshot of a retried test always shows the last attempt #11614

Closed

sheremet-va previously approved these changes Oct 19, 2026

View reviewed changes

Merge remote-tracking branch 'upstream/main' into attempts-api-prototype

eed88dc

# Conflicts: # packages/vitest/src/runtime/runner/run.ts

hi-ogawa dismissed sheremet-va’s stale review via eed88dc October 19, 2026 09:55

hi-ogawa requested a review from sheremet-va October 19, 2026 10:09

sheremet-va merged commit 21be412 into main Oct 19, 2026
29 checks passed

sheremet-va deleted the attempts-api-prototype branch October 19, 2026 14:39

hi-ogawa mentioned this pull request Oct 21, 2026

feat: expose current attempt index on task result #11680

Draft

Navigation Menu

Repository navigation

feat: expose per-retry/repeat attempt result - #11232

feat: expose per-retry/repeat attempt result#11232
sheremet-va merged 68 commits into
mainfrom
attempts-api-prototype

hi-ogawa commented Sep 11, 2026 •

edited

Loading

Uh oh!

sheremet-va commented Oct 19, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

hi-ogawa commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

sheremet-va commented Oct 19, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hi-ogawa commented Sep 11, 2026 •

edited

Loading