Skip to content

Add AI Trace plugin for AI-assisted debugging#5432

Open
DenysKuchma wants to merge 2 commits into4.xfrom
feature/5425-ai-trace-plugin
Open

Add AI Trace plugin for AI-assisted debugging#5432
DenysKuchma wants to merge 2 commits into4.xfrom
feature/5425-ai-trace-plugin

Conversation

@DenysKuchma
Copy link
Collaborator

Summary

Adds new aiTrace plugin that generates AI-friendly trace files for debugging failing tests with AI agents.

What it does

  • Generates structured markdown trace files for each test
  • Captures screenshots, HTML, ARIA snapshots, console logs
  • Links to HAR files and Playwright traces
  • Optimized for AI analysis

Configuration

plugins: {
  aiTrace: {
    enabled: true,
    deleteSuccessful: true,  // only keep failing tests
    ignoreSteps: [/^grab/, /^wait/]
  }

Output

For each test creates trace_/ directory with:

  • trace.md - AI-friendly execution log
  • 0000_screenshot.png - page screenshot
  • 0000_page.html - full HTML
  • 0000_aria.txt - accessibility tree
  • 0000_console.json - browser console logs

Use Cases

  • AI Agents: Provide context to Claude Code/GitHub Copilot for debugging
  • CI/CD: Attach traces as build artifacts for failed tests
  • Bug Reports: Share comprehensive reproduction artifacts
  • Local Debugging: Quick investigation without re-running tests

Closes #5425

@DenysKuchma DenysKuchma requested a review from DavertMik February 4, 2026 21:38
fs.writeFileSync(path.join(dir, htmlFile), html)
stepData.artifacts.html = htmlFile
} catch (err) {
output.debug(`aiTrace: Could not capture HTML: ${err.message}`)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think if we could consolidate something like this https://github.com/codeceptjs/CodeceptJS/blob/4.x/lib/plugin/analyze.js#L2, maybe it's easier for debugging 😄

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'll think about it while continuing to work on the related task #5426

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants