Skip to content

[performance] Likely environment noise: 5 benchmarks show 33%-328% regressions with no code cause #53729

Description

@github-actions

📊 Performance Regression Pattern Detected (Likely Environment Noise)

On 2026-08-18, 5 of 9 tracked benchmarks showed apparent regressions of 33%–328% versus their historical averages, while 3 improved and 1 warned. Critically, the "regressed" set includes unrelated micro-benchmarks with no shared code path, e.g. ExtractWorkflowNameFromFile (+328%, simple filename parsing) and FindIncludesInContent (+33%, single regex scan over static content) — these have no reasonable causal link to compiler changes.

📈 Detailed Regression Data
Benchmark Current Historical Avg Change
CompileComplexWorkflow 39.17ms 12.30ms +218.4%
CompileMCPWorkflow 3.71ms 1.01ms +265.7%
ParseWorkflow 780.90μs 372.09μs +109.9%
ExtractWorkflowNameFromFile 172.22μs 40.23μs +328.1%
FindIncludesInContent 247ns 185ns +33.5%

Historical data points used: 5 (last recorded prior entry was 2026-07-06, ~6 weeks before this run — a large gap in the daily cadence).

git log --since="48 hours ago" over pkg/workflow, pkg/cli, pkg/parser shows no relevant commits in the window that could explain a 2–4x slowdown across otherwise-unrelated benchmarks.

💡 Assessment

This pattern — broad, simultaneous, large-magnitude slowdowns across unrelated benchmarks, no corresponding code changes, and a low sample size (n=3 iterations per benchmark) — is much more consistent with noisy/shared CI runner conditions than a genuine performance regression in the codebase.

🔧 Recommended Actions

  1. Re-run make bench-performance on a quieter/dedicated runner to confirm whether the slowdown reproduces.
  2. Consider increasing -benchtime / iteration count in bench-performance to reduce sensitivity to transient runner noise.
  3. If the gap in daily history entries (2026-07-06 → 2026-08-18) reflects workflow skips, verify the daily performance workflow's activation/skip logic is still triggering correctly.
  4. No code-level performance investigation is recommended until reproduced on a controlled environment.

Automatically generated by Daily CLI Performance workflow — Run ID: 32151688389

Generated by ⚡ Daily CLI Performance Agent · auto · 34.9 AIC · ⌖ 9.95 AIC · ⊞ 18.2K ·

  • expires on Aug 20, 2026, 7:14 AM UTC-08:00

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions