[delight] User Experience Analysis Report - 2026-08-21

[github-actions[bot]]

Executive Summary

Today's analysis focused on:

• 2 documentation file(s): weekly-update blog post, "Meet the Workflows: Interactive & ChatOps" blog post
• 0 CLI command(s) — gh-aw binary was not buildable in this sandbox (network-restricted; Go toolchain download blocked), so CLI help text could not be evaluated this run
• 2 workflow message configuration(s): necromancer.md, ci-doctor.md
• 1 validation file: pkg/workflow/template_injection_validation_benchmark_test.go (benchmark test, not user-facing — no actionable finding here; substituted a second workflow-messages finding instead)

Overall Quality: Professional overall, with one workflow (ci-doctor.md) leaning noticeably more casual/thematic than its peers, and documentation that is otherwise clear and well-linked.

Key Finding: ci-doctor.md's status messages use heavier "hospital" role-play language (four emoji across three short messages, phrases like "Doctor needs assistance...") that reads less professionally than comparable workflows such as necromancer.md, which uses restrained emoji and clearer status language.

Quality Highlights ✅

Example 1: Weekly update blog post structure

• File: docs/src/content/docs/blog/2026-06-29-weekly-update.md
• What works well: Clear heading hierarchy (H2 per topic), each item links directly to the source PR, includes a comparison table for version bumps, and closes with a concrete "Agent of the Week" section with measurable details (AIC cost, model used, API call count). This gives enterprise readers a scannable, evidence-backed update.
• Quote/Reference: "The two successful runs consumed around 24 AIC each, used gpt-5.4 for analysis, and made 13 GitHub API calls to gather workflow context before synthesizing findings."

Example 2: Necromancer workflow messaging

• File: .github/workflows/necromancer.md
• What works well: Messages use a single themed emoji per message (not stacked), remain short, and clearly communicate state — started/success/failure — without excessive personality overwhelming the information.
• Quote/Reference: run-success: "✅ [{workflow_name}]({run_url}) fortified this PR with fresh regression coverage."

Improvement Opportunities 💡

High Priority

Opportunity 1: CI Doctor workflow messages overuse role-play framing - Single File Improvement

• File: .github/workflows/ci-doctor.md (lines 34-37)
• Current State: Four different emoji (🩺, 🏥, 🩺, 🏥) across three status lines, plus phrases like "reporting for duty!", "examining the patient", "Prescription issued! 💊", and "Doctor needs assistance..." for a failure state.
• Issue: The "hospital" metaphor is layered on every single message rather than used once for flavor, and the failure message ("Doctor needs assistance...") anthropomorphizes the failure instead of stating what happened or what to do next — reducing clarity for engineers scanning PR/issue comments under time pressure.
• User Impact: Enterprise users skimming CI failure notifications need to quickly identify severity and next steps; heavy theming adds a decoding step and can read as unprofessional in shared/external-facing repos.
• Suggested Change: Trim to one thematic emoji per message maximum, and make the failure message state the actual condition (e.g., "encountered an error while diagnosing this failure — see the run log for details") rather than "Doctor needs assistance."
• Design Principle: Professional Communication (balanced emoji use) + Trust and Reliability (clear, actionable failure messaging)

Medium Priority

Opportunity 2: Weekly update blog post could clarify "cookie" label meaning

• File: docs/src/content/docs/blog/2026-06-29-weekly-update.md
• Minor/no code change needed this run — deferred, no line-level issue found beyond Opportunity 1.

Files Reviewed

Documentation

• docs/src/content/docs/blog/2026-06-29-weekly-update.md - Rating: ✅
• docs/src/content/docs/blog/2026-01-13-meet-the-workflows-interactive-chatops.md - Rating: ✅

CLI Commands

• Not evaluated this run (build blocked by sandbox network restrictions)

Workflow Messages

• .github/workflows/necromancer.md - Rating: ✅
• .github/workflows/ci-doctor.md - Rating: ⚠️ Needs Minor Work

Validation Code

• pkg/workflow/template_injection_validation_benchmark_test.go - Rating: ✅ (not user-facing; no error-message content to review)

Metrics

• Files Analyzed: 5
• Quality Distribution:
  • ✅ Professional: 4
  • ⚠️ Needs Minor Work: 1
  • ❌ Needs Significant Work: 0

🎯 Actionable Tasks

Here are 1 targeted improvement task, affecting a single file:

Task 1: Tone Down Role-Play Overload in CI Doctor Status Messages

File to Modify: .github/workflows/ci-doctor.md

Current Experience

Lines 34-37 define the messages block for CI Doctor. Every one of the four messages (footer, run-started, run-success, run-failure) layers hospital-themed emoji and phrasing on top of each other:

footer: "> 🩺 *Diagnosis provided by [{workflow_name}]({run_url})*{ai_credits_suffix}{history_link}"
run-started: "🏥 CI Doctor reporting for duty! [{workflow_name}]({run_url}) is examining the patient on this {event_type}..."
run-success: "🩺 Examination complete! [{workflow_name}]({run_url}) has delivered the diagnosis. Prescription issued! 💊"
run-failure: "🏥 Medical emergency! [{workflow_name}]({run_url}) {status}. Doctor needs assistance..."

Quality Issue

Design Principle: Professional Communication (balanced emoji use) and Trust and Reliability (clear, actionable error messages)

Stacking emoji and metaphor language ("examining the patient", "Prescription issued!", "Doctor needs assistance...") on every message increases decoding effort for engineers scanning comment threads, especially the failure message, which never states what actually happened.

Proposed Improvement

Keep the mascot personality in the footer only, and use plain, informative language for the operational states — especially failure, which should hint at where to look for details.

Before:

footer: "> 🩺 *Diagnosis provided by [{workflow_name}]({run_url})*{ai_credits_suffix}{history_link}"
run-started: "🏥 CI Doctor reporting for duty! [{workflow_name}]({run_url}) is examining the patient on this {event_type}..."
run-success: "🩺 Examination complete! [{workflow_name}]({run_url}) has delivered the diagnosis. Prescription issued! 💊"
run-failure: "🏥 Medical emergency! [{workflow_name}]({run_url}) {status}. Doctor needs assistance..."

After:

footer: "> 🩺 *Diagnosis provided by [{workflow_name}]({run_url})*{ai_credits_suffix}{history_link}"
run-started: "🩺 [{workflow_name}]({run_url}) is examining this {event_type}..."
run-success: "✅ [{workflow_name}]({run_url}) completed its diagnosis of this {event_type}."
run-failure: "⚠️ [{workflow_name}]({run_url}) {status} while diagnosing this {event_type} — check the run log for details."

Why This Matters

• User Impact: Faster, clearer scanning of CI status in PR/issue comments; failure state now points to an actionable next step (the run log).
• Quality Factor: Professional Communication, Trust and Reliability
• Frequency: These messages post on every CI Doctor run, so the improvement compounds across all future invocations.

Success Criteria

• Changes made to .github/workflows/ci-doctor.md only
• Failure message includes an actionable pointer (e.g., "check the run log")
• Emoji count reduced to one per message
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: .github/workflows/ci-doctor.md
• No changes to other files required
• Can be completed independently

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• storage.googleapis.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "storage.googleapis.com"

See Network Configuration for more information.

📊 User experience analysis by Delight · auto · 39.3 AIC · ⌖ 5.04 AIC · ⊞ 12.3K · ◷

• expires on Aug 24, 2026, 7:04 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Delight.

A newer discussion is available at Discussion #54843.