Skip to content

feat: add M2 performance benchmark harness#20

Merged
anandh8x merged 2 commits into
mainfrom
feat/m2-perf-bench
Jun 3, 2026
Merged

feat: add M2 performance benchmark harness#20
anandh8x merged 2 commits into
mainfrom
feat/m2-perf-bench

Conversation

@anandh8x

@anandh8x anandh8x commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add scripts/perf-bench.ts for cold start, deterministic TTFT, stream overhead, and RSS measurements.
  • Add perf:bench and perf:smoke package scripts plus performance docs.
  • Add a CI Performance Smoke job that builds the binary, emits GitHub warning annotations for threshold drift, and uploads the JSON report.

Validation

  • bun run test
  • bun run typecheck
  • bun run build
  • bun run smoke:build
  • bun run perf:smoke
  • bun run package:release
  • git diff --check

Note

  • Local perf:smoke passed and intentionally warned that cold-start p95 is above the PRD target of 300 ms. The new job reports that as a warning, not a blocking failure.

@gnanam1990 gnanam1990 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Hermes Agent rereview

Verdict: Approve

This PR had not yet been reviewed by gnanam1990, so I performed a full review of the current PR diff against origin/main.

Existing review/comment context considered

  • Reviews: none found.
  • Issue comments: none found.

Scope reviewed

  • Adds scripts/perf-bench.ts for cold-start, TTFT, stream overhead, and RSS benchmark reporting.
  • Adds helper tests in tests/perf-bench.test.ts.
  • Adds perf:bench / perf:smoke package scripts, performance CI job, and docs.

Validation

  • npx --yes bun install --frozen-lockfile
  • npx --yes bun test ✅ (138 passing)
  • npx --yes bun run typecheck
  • npx --yes bun run build
  • Targeted performance smoke: npx --yes bun run perf:smoke
    • cold start p95 77.04 ms; TTFT p95 0.48 ms; RSS peak 67.81 MB; warnings: none
  • GitHub checks observed passing for Performance Smoke and all Smoke matrix jobs ✅

I did not find blocking correctness, security, or maintainability issues. The benchmark code handles CLI/env parsing errors, emits CI warnings without failing by default, writes reports safely under caller-provided paths, and has useful unit plus end-to-end coverage.

@Vasanthdev2004

Copy link
Copy Markdown
Collaborator

Blockers

None found.

Non-Blocking

None.

Looks Good

  • Clean M2 performance harness: cold-start, deterministic TTFT/stream overhead, RSS metrics, JSON report output, threshold warnings, and optional --fail-on-warning behavior are all covered.
  • CI integration is safe: the performance job builds first, emits GitHub warning annotations for drift, uploads the report, and only fails for real benchmark/runtime failures unless --fail-on-warning is explicitly used.
  • I also tested this on top of latest origin/main because the PR base was behind after feat: add Zero session event store #21 merged; the merge was clean and validation stayed green.
  • PR branch validation passed: install, test, typecheck, build, smoke build, perf smoke, package release, checksum verification, and diff whitespace check.
  • Latest-main test-merge validation passed: typecheck, full tests, build, smoke build, perf smoke, package release, checksum verification, and diff whitespace check.
  • Test-merge result: 156 pass / 0 fail / 580 expect calls.
  • Perf smoke on the latest-main merge produced no warnings: cold-start p95 220.19 ms, TTFT p95 1.33 ms, RSS peak 91.27 MB.
  • Scope is focused: 6 files, +817/-1.

Verdict: Approve — solid M2 benchmark/CI foundation and safe to merge after the stale-base check.

@Vasanthdev2004 Vasanthdev2004 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Solid M2 performance benchmark/CI foundation; PR branch and latest-main test-merge validation passed.

@anandh8x
anandh8x merged commit 0ac9009 into main Jun 3, 2026
4 checks passed
@Vasanthdev2004
Vasanthdev2004 deleted the feat/m2-perf-bench branch June 28, 2026 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants