Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Boot-to-end perf baseline — 2026-05-08 — post-pr-19-pre-adr-0027

Generated by `tools/perf-harness.sh` — multi-run aggregation of the kernel's
`boot-to-end elapsed = X ns` emission (P10 from the [2026-05-06 Track D
review](../reviews/code-reviews/2026-05-06-full-tree/track-d-performance.md)).

## Inputs

| Field | Value |
|-------|-------|
| Run timestamp (UTC) | `2026-05-08T09:57:08Z` |
| Iterations requested | 20 |
| Iterations valid | 20 |
| Iterations failed | 0 |
| Per-run timeout | 5 s |
| Build profile | debug |
| Kernel ELF | `target/aarch64-unknown-none/debug/tyrne-bsp-qemu-virt` |
| Git HEAD | `aa7e6c5` on `p10-wall-clock-bench-harness` |
| QEMU | `QEMU emulator version 10.2.2` |
| Host `uname -a` | `Darwin MacBookPro.hgw.local 24.6.0 Darwin Kernel Version 24.6.0: Wed Nov 5 21:30:23 PST 2025; root:xnu-11417.140.69.705.2~1/RELEASE_X86_64 x86_64` |
| Wall-clock (full harness run) | 101 s |

## Methodology

Each iteration invokes `tools/run-qemu.sh` under a per-run watchdog;
QEMU emits the boot trace through to `tyrne: all tasks complete` plus
the `boot-to-end elapsed = X ns` line, then halts in WFI. The watchdog
kills the QEMU process after the per-run timeout (the kernel never
exits on its own). The integer ns delta is parsed out of stdout.

Counter source: the kernel's `now_ns()` (`hal::Timer`) reads the EL1
virtual generic-timer counter and converts to nanoseconds via the
cached `CNTFRQ_EL0` resolution. Under QEMU TCG the counter advances
based on emulated instructions rather than wall-clock time, so
variance reflects translation-cache behaviour and host scheduler
jitter, not real hardware performance.

Statistics are computed across the valid samples only. Percentile
convention is *nearest-rank* (1-indexed; `idx = ceil(p/100 * n)`).
Stddev is the population formula (`n` divisor) — descriptive.

## Metric — boot-to-end elapsed (nanoseconds)

| Statistic | ns | ms |
|-----------|---:|---:|
| min | 3,862,000 | 3.862 |
| p10 | 3,884,000 | 3.884 |
| p50 | 4,642,000 | 4.642 |
| p90 | 5,584,000 | 5.584 |
| p99 | 6,558,000 | 6.558 |
| max | 6,558,000 | 6.558 |
| mean | 4,711,000 | 4.711 |
| stddev | 708,746 | 0.709 |

## Raw samples

One ns value per line, in iteration order (NOT sorted):

```
3949008
3862000
5482000
6558000
3884000
4930000
4370992
5085008
5698000
4024992
5584000
3888992
4339008
4834000
4056000
4642000
4314000
4968000
4828000
4922000
```

## Verdict

Baseline only — no proposal under measurement. Cite the band above
(p10 / p50 / p90) when comparing later changes against this snapshot.
Single-run boot-to-end claims in PR bodies should be replaced with a
fresh harness run when a non-trivial perf-relevant change lands; see
[`docs/standards/infrastructure.md`](../../standards/infrastructure.md)
§"Performance harness".
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,8 @@ The bookkeeping closes byte-exactly: +12 + 216 = +228 = section-level delta. Zer

The 11 P-numbered proposals from Track D's [2026-05-06 paper review](../code-reviews/2026-05-06-full-tree/track-d-performance.md) remain queued (P3 partially landed by γ; P1 / P10 / P4 are the highest-ROI near-term picks and ripe for an explicit cycle once B2's MMU work surfaces a measurement target).

**Post-amendment update (2026-05-08, on its own PR):** P10 has now landed as [`tools/perf-harness.sh`](../../../../tools/perf-harness.sh); the first measured replacement for the "~4–6.5 ms typical" anecdote above is recorded in [`perf-baseline-2026-05-08-post-pr-19-pre-adr-0027.md`](../../reports/perf-baseline-2026-05-08-post-pr-19-pre-adr-0027.md) (debug build, 20 iterations, p10=3.884 ms / p50=4.642 ms / p90=5.584 ms on QEMU TCG). The single-run claims throughout this baseline are preserved as the historical record of what was true on the day; new perf claims after this point cite the harness band per [`infrastructure.md` §"Performance harness"](../../../standards/infrastructure.md#performance-harness).

### Cross-references

- **Business retrospective:** [`2026-05-07-B1-closure.md`](../business-reviews/2026-05-07-B1-closure.md).
Expand Down
4 changes: 4 additions & 0 deletions docs/roadmap/current.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ A short pointer file updated as work progresses. For the full plan see [`phases/

---

> **2026-05-08 update — P10 wall-clock harness landed; boot-to-end band re-quoted.** [`tools/perf-harness.sh`](../../tools/perf-harness.sh) now wraps `tools/run-qemu.sh` in a multi-run aggregation loop with `min / p10 / p50 / p90 / p99 / max / mean / stddev` reporting (P10 from the [2026-05-06 Track D review](../analysis/reviews/code-reviews/2026-05-06-full-tree/track-d-performance.md), promoted by the [2026-05-07 multi-axis review §D2](../analysis/reviews/code-reviews/2026-05-07-pr-12-to-17-multi-axis-review/track-d-perf.md) ahead of B2 ADR-0027 implementation). First measured baseline at HEAD `aa7e6c5` (debug build, 20 iterations, 5 s per-run timeout, QEMU TCG): **p10=3.884 ms / p50=4.642 ms / p90=5.584 ms / p99=6.558 ms** with mean 4.711 ms, stddev 0.709 ms — see [`docs/analysis/reports/perf-baseline-2026-05-08-post-pr-19-pre-adr-0027.md`](../analysis/reports/perf-baseline-2026-05-08-post-pr-19-pre-adr-0027.md). The band brackets the previous "~4–6.5 ms typical" anecdote tightly, but it is now a *measured* band on this host rather than an order-of-magnitude observation; B2 perf changes will be evaluated against it. Single-run boot-to-end claims in PR bodies are deprecated per the new [`infrastructure.md` §"Performance harness"](../standards/infrastructure.md#performance-harness) policy.

---

> **2026-05-07 update — B1 truly closed; T-015 (B2 prep follow-on) Done.** T-014 promoted `In Review → Done` and a fresh closure trio (business + consolidated security + performance baseline) landed today, replacing the [2026-04-28 trio](../analysis/reviews/business-reviews/2026-04-28-B1-closure.md)'s load-bearing role. The 2026-04-28 trio remains the historical record of "what we believed at PR #10 merge"; the 2026-05-07 trio records "what is true post-T-014". Smoke-confirmed end-to-end (full demo trace through `tyrne: all tasks complete` plus `boot-to-end elapsed = ... ns`; ~4–6.5 ms typical on QEMU-default Cortex-A72; `-d int,unimp,guest_errors` empty). 152/152 host tests + 152/152 miri clean at PR #16 merge; **post-T-015** (this PR #17) **158/158 host tests + 158/158 miri clean** (5 new IPC `cancel_recv` unit tests + 1 scheduler `ipc_recv_and_yield_deadlock_rolls_back_endpoint_state` regression test; existing T-007 Deadlock test gained an endpoint-state assertion). T-015 closed an open B2-prep follow-on (ADR-0032's `ipc_cancel_recv` recovery primitive — symmetric scheduler+endpoint rollback when `SchedError::Deadlock` returns). **B2 prep (ADR-0027 kernel virtual memory layout) is now the active implementation thread.**

---
Expand Down
27 changes: 27 additions & 0 deletions docs/standards/infrastructure.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,33 @@ CI is expected to be set up early in Phase 4 (Rust toolchain + workspace skeleto
- `qemu-system-aarch64` on the Linux runner for smoke tests.
- Real-hardware jobs (Raspberry Pi lab, when it exists) are self-hosted runners, off the PR hot path, running on release cadence.

## Performance harness

`tools/perf-harness.sh` is the canonical source for boot-to-end timing claims. It wraps `tools/run-qemu.sh` in an iteration loop with a per-run watchdog, parses the kernel's `boot-to-end elapsed = X ns` emission out of each run's serial output, and prints `min / p10 / p50 / p90 / p99 / max / mean / stddev` in both ns and ms. Maintainer-launched only; not yet wired into CI (matches the QEMU smoke convention above — promotion to a CI gate is a B2-or-later follow-up alongside the smoke job).

### Usage

```text
tools/perf-harness.sh # 20 iterations, debug build
tools/perf-harness.sh --iterations=K --timeout=SECONDS # tune iteration count + per-run watchdog
tools/perf-harness.sh --release # use the release ELF (forwarded to run-qemu.sh)
tools/perf-harness.sh --quiet # suppress per-iteration progress
tools/perf-harness.sh --report=CONTEXT # also emit a markdown report under
# docs/analysis/reports/perf-baseline-YYYY-MM-DD-CONTEXT.md
```

A run aborts non-zero if fewer than 50 % of iterations produced a valid sample — that threshold is treated as environmental (kernel image missing, QEMU not in PATH, host under heavy load) rather than a measurement worth aggregating.

### Reporting discipline

- **Cite the band, not a single sample.** When a PR's commentary needs a boot-to-end figure, run the harness and quote the `p10 / p50 / p90` triple plus the iteration count. A solitary `boot-to-end elapsed = X ns` from a single QEMU launch is not a load-bearing measurement; QEMU TCG's translation-cache behaviour gives ~15-30 % run-to-run variance and a single sample can fall anywhere in the band.
- **Single-run anecdotes from before this harness landed are preserved as historical record.** The 2026-04-21 / 2026-04-28 / 2026-05-07 perf reviews quote single-run figures; those numbers are not retroactively replaced — but every *new* perf claim cites a harness band.
- **Baseline reports under `docs/analysis/reports/perf-baseline-*.md`** are append-only artefacts. Re-baselines after a perf-relevant change land as fresh reports with a new context slug; old reports stay in place as the historical record.

### Counter caveat

The harness measures the kernel's `now_ns()` delta. Under QEMU TCG that counter advances based on emulated instructions, so the band reflects translation-cache variance plus host-scheduler jitter rather than wall-clock time on real hardware. The numbers are useful for *relative* regression detection across a tight window of commits on the same host; they are not predictive of boot-time on real ARM silicon. When that question becomes load-bearing the harness gains a `--hardware` mode or the measurement moves to a self-hosted Pi runner — neither is in scope for v1.

## Supply-chain security

### `cargo-vet`
Expand Down
Loading
Loading