Add SQLite persistence and run history to the Dashboard

[James Newton-King (JamesNK)]

Specification: Dashboard persistence

Fixes #4256

Description

Dashboard resources and telemetry previously existed only for the lifetime of the Dashboard process. This change adds SQLite-backed persistence so users can inspect completed application runs, compare telemetry before and after code changes, or keep a standalone Dashboard's data across restarts.

Highlights

• Adds versioned, normalized SQLite storage for resource snapshots, console logs, structured logs, traces, spans, events, links, metrics, dimensions, histogram data, and exemplars. Telemetry is stored relationally rather than as opaque OTLP payloads.
• Adds three persistence modes:
  • None creates a temporary database for one Dashboard process and remains the standalone Dashboard default.
  • Run creates a persistent database for each Dashboard process, retains up to ten runs per application, and is the AppHost Dashboard default.
  • Resume reuses one persistent database across Dashboard restarts without exposing run selection.
• Adds a run selector to the Dashboard header. Users can switch between Live run and completed runs without reloading the browser. Historical runs are opened read-only, mutation controls are disabled, and metric views use the latest stored timestamp as their fixed end time.
• Adds run metadata, exclusive ownership locks, schema compatibility checks, abandoned-run handling, and lock-aware pruning. Historical databases are pooled and shared across Dashboard circuits while selected.
• Moves filtering, searching, paging, aggregation, field-value lookup, and retention into SQLite. Batched writes, transactions, connection pooling, normalized identities, targeted caches, trace summaries, and metric rollups support high-volume ingestion and bounded UI queries.
• Passes the normalized application name, data directory, and persistence mode from AppHost, and adds --application-name and --persistence to aspire dashboard run.
• Uses Microsoft.Data.Sqlite with the transitive SQLitePCLRaw.bundle_e_sqlite3 and SQLitePCLRaw.lib.e_sqlite3 native packages, including the required native assets in the managed Dashboard bundle.
• Exercises the SQLite repositories through shared resource and telemetry repository tests, with additional coverage for run lifecycle, retention, schema compatibility, read-only history, run selection, and persistence configuration.

Persistence scenarios

Temporary standalone Dashboard

Run the standalone Dashboard without persistence configuration:

aspire dashboard run

The default None mode is useful for a single development or diagnostic session. Its temporary database is deleted when the Dashboard stops.

Compare AppHost runs

Start an AppHost normally. Its Dashboard defaults to Run, so each AppHost start creates a separate run database. After changing application code and restarting the AppHost, use the header selector to compare resources and telemetry from Live run with completed runs.

Resume a standalone Dashboard

Give a standalone Dashboard a stable application name and select Resume:

aspire dashboard run --application-name my-app --persistence Resume

The Dashboard reopens the same database after restart. Container deployments must mount ASPIRE_DASHBOARD_DATA_DIRECTORY from persistent storage and reuse the same application name, directory, and persistence mode.

Screenshots / Recordings

Security considerations

Persisted Dashboard data can contain sensitive application-supplied values. Resource properties marked sensitive are stored without redaction or encryption, although their sensitivity marker is retained and current and historical values remain masked in the UI.

The database has no independent authorization or encryption layer. Operators must ensure that ASPIRE_HOME or Dashboard:Data:Directory, plus backups and copies, have restrictive filesystem access controls that prevent unauthorized users from reading or modifying persisted data. The Dashboard does not set or validate specific ACLs or Unix file modes.

This change does not add a network endpoint. Network ingestion, endpoint authentication and authorization, transport security, and data-read APIs are existing Dashboard concerns and are documented separately.

Validation

• dotnet test --project tests/Aspire.Dashboard.Tests/Aspire.Dashboard.Tests.csproj --configuration Release --no-launch-profile -- --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"
• dotnet test --project tests/Aspire.Dashboard.Components.Tests/Aspire.Dashboard.Components.Tests.csproj --no-launch-profile -- --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"
• dotnet build src/Aspire.Dashboard/Aspire.Dashboard.csproj --no-restore

Checklist

• Is this feature complete?
  • Yes. Ready to ship.
  • No. Follow-up changes expected.
• Are you including unit tests for the changes and scenario tests if relevant?
  • Yes
  • No
• Did you add public API?
  • Yes
    • If yes, did you have an API Review for it?
      • Yes
      • No
    • Did you add <remarks /> and <code /> elements on your triple slash comments?
      • Yes
      • No
  • No
• Does the change make any security assumptions or guarantees?
  • Yes
    • If yes, have you done a threat model and had a security review?
      • Yes
      • No
  • No

[github-actions]

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 18768

Or

• Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 18768"

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum diff size. Try reducing the number of changed files and lines, and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum diff size. Try reducing the number of changed files and lines, and requesting a review from Copilot again.

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[Copilot]

Pull request overview

Copilot reviewed 138 out of 139 changed files in this pull request and generated 5 comments.

Files not reviewed (1)

• src/Aspire.Dashboard/Resources/Dialogs.Designer.cs: Generated file

[Copilot]

Pull request overview

Copilot reviewed 138 out of 139 changed files in this pull request and generated 5 comments.

Files not reviewed (1)

• src/Aspire.Dashboard/Resources/Dialogs.Designer.cs: Generated file

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[David Pine (IEvangelist)]

PR Testing Report

PR Information

• PR Number: #18768
• Title: Add SQLite persistence and run history to the Dashboard
• Head Commit: 08cfaac44c192ea3525813ac8835cdbfccef718c
• Fixes: Aspire Dashboard - Persist dashboard data #4256
• Tested At: 2026-07-28 (local, Windows)
• Execution target: Local, isolated temp dir (-InstallPath -SkipExtension -SkipPath)

Artifact Version Verification

• Expected Commit: 08cfaac4…
• Installed CLI Version: 13.5.0-pr.18768.g08cfaac4
• Status: ✅ Verified (version contains PR head short SHA g08cfaac4)

Changes Analyzed (279 files)

Change Categories

• CLI — src/Aspire.Cli/Commands/DashboardRunCommand.cs: adds --application-name and --persistence to aspire dashboard run
• Dashboard — SQLite persistence (resources + normalized telemetry), run-history selector (DashboardRunSelect), read-only historical runs, retention/locking, DashboardOptions/validation, chart/metric caching
• Hosting/playground — AppHost passes normalized app name + data directory + persistence mode (defaults to Run); Stress + TestShop AppHosts updated
• Tests — extensive (CLI, Dashboard, Components, Hosting, shared telemetry repos)
• VS Code extension — none
• CI infrastructure — none

Persistence modes (from PR)

• None — temporary DB for one Dashboard process, deleted on stop. Standalone default.
• Run — persistent DB per Dashboard process, retains up to 10 runs/app. AppHost default. Enables run selector.
• Resume — reuse one persistent DB across restarts. No run selector.

Test Scenarios Executed

Scenario 1: CLI help shows new options

Objective: aspire dashboard run exposes the new options. Coverage: Happy path. Status: ✅ Passed

• aspire dashboard run --help lists:
  • --application-name <application-name> — "The application name displayed in the dashboard and used to scope persisted dashboard data"
  • --persistence <persistence> — "The dashboard data persistence mode: None, Run, or Resume"

---

Scenario 2: Standalone dashboard, None mode (default)

Objective: Default standalone mode uses a temporary DB that is cleaned up on stop. Coverage: Happy path. Status: ✅ Passed

Steps & evidence:

1. aspire dashboard run --allow-anonymous (ran on alt ports 18899/14317/14318 to avoid a pre-existing dashboard on the default ports).
2. Log: Dashboard run store initialized with persistence mode 'None'. Run directory: '…\Temp\aspire-dashboard-<rand>'. Database path: '…\dashboard.db'.
3. While running, temp DB present in WAL mode: dashboard.db, dashboard.db-shm, dashboard.db-wal.
4. Dashboard renders (screenshot dashboard-none-mode.png); no run selector in header (correct for None).
5. Graceful shutdown (Ctrl+Break) → exit code 0 → temp DB directory deleted (DB_DIR_EXISTS_AFTER_STOP = False).

Observation: A forced kill (not graceful) leaves the temp directory behind — expected, since cleanup runs on graceful shutdown.

---

Scenario 3: Standalone dashboard, Resume mode

Objective: Same persistent DB reused across restarts, scoped by application name. Coverage: Happy path. Status: ✅ Passed

Steps & evidence:

1. aspire dashboard run --application-name my-app --persistence Resume with ASPIRE_DASHBOARD_DATA_DIRECTORY set to an explicit dir. Two sequential graceful runs.
2. DB path is deterministic and app-scoped: …\data\my-app-97768f84a7542bed\dashboard.db.
3. DB file persists after graceful stop (401 KB after run 1).
4. Restart reuses the identical DB path (SAME_DB_PATH_ACROSS_RESTART = True).
5. Header displays the application name "my-app" (page title my-app structured logs); no run selector (correct for Resume). Screenshot dashboard-resume-mode.png.

---

Scenario 4: AppHost Run mode + run history (headline feature)

Objective: AppHost runs persist per-run; the run selector lets you open completed runs read-only. Coverage: Happy path. Status: ✅ Passed

Steps & evidence:

1. Created aspire-starter (PrRunHistory) from the PR hive; AppHost dashboard defaults to Run.
2. Run 1 (aspire run --detach --format Json) → 2 resources Running (apiservice, webfrontend). Header shows green "● Live run" selector (screenshot run1-live-resources.png).
3. On-disk Run layout under the app-scoped folder PrRunHistory-<hash>\runs\<timestamp>\:
   • dashboard.db (+ -shm/-wal), a <timestamp>.lock (exclusive ownership), and run.json metadata (SchemaVersion: 17, RunId, StartedAtUtc, EndedAtUtc, CleanShutdown, ApplicationName, DatabaseFileName).
4. aspire stop (graceful) → lock released; run 1 folder retained.
5. Run 2 → now two run folders on disk; only the active run is locked.
6. Header run selector dropdown lists "✓ Live run" and the completed "11:25:47 AM" run (screenshot run2-selector-open.png).
7. Selecting "11:25:47 AM" switches to run 1's data (its original start times 11:25:45/11:25:50), the header indicator turns red, and the resource action controls are disabled (read-only). Screenshot run2-historical-final.png.

Observation (minor, non-blocking): After aspire stop, run 1's run.json still showed EndedAtUtc: null and CleanShutdown: false even though the exclusive lock was released. The run is treated as an unlocked/abandoned run and remains fully browsable read-only, but the "clean shutdown / ended" metadata was not finalized on stop. Worth confirming with the author whether a graceful aspire stop is expected to mark the run as cleanly ended.

---

Scenario 5: Invalid --persistence value (unhappy path)

Objective: Invalid persistence mode fails safely. Coverage: Unhappy path. Status: ✅ Passed

• aspire dashboard run --persistence Bogus → CLI prints ❌ Dashboard configuration is invalid. Check the provided options and try again. and exits with code 12.
• Dashboard log pinpoints the cause: Failed to parse dashboard persistence mode 'Bogus'. Possible values: None, Run, Resume. (managed process exits 101; dashboard does not start).

Expected unhappy-path outcome: Clear validation error, non-zero exit, no dashboard startup. ✅ Met.

Summary

Scenario	Status	Notes
1. CLI help shows new options	✅ Passed	--application-name, --persistence present with correct descriptions
2. Standalone None (default)	✅ Passed	Temp WAL DB created; deleted on graceful stop; no run selector
3. Standalone Resume	✅ Passed	Deterministic app-scoped DB; persists + reused across restart; app name in header
4. AppHost Run + run history	✅ Passed	Per-run DB/lock/metadata; selector shows Live + completed; historical run read-only
5. Invalid --persistence	✅ Passed	Precise validation error, exit 12, no startup

Overall Result

✅ PR VERIFIED — All 5 scenarios passed. The new aspire dashboard run --application-name/--persistence options, the three persistence modes (None/Run/Resume), and the run-history selector with read-only historical runs all behave as described.

Recommendations

• Consider confirming the run.json finalization on graceful aspire stop (Scenario 4 observation): the exclusive lock is released but EndedAtUtc/CleanShutdown stay unset, so completed runs surface as abandoned rather than cleanly ended. Functionally harmless (runs are still browsable), but the metadata may be worth tidying.

---

Tested with the "Dogfood this PR" CLI in an isolated temp directory; dashboard UI verified with Playwright/Chromium. No changes made to the repository.

[David Pine (IEvangelist)]

Code review focused on issues not already covered by the existing review threads, across the SQLite persistence layer, run lifecycle/config/CLI, and the read-only invariant. 3 findings: one reachable bug (Resume-mode console-log duplication), one latent read-only defense-in-depth gap, and one low-severity startup-robustness issue. Everything else reviewed (run-store locking/retention math, data-source pool ref-counting/disposal, config validation, CLI --application-name/--persistence wiring, SQL parameterization/transactions/disposal, batch-insert chunking, and the rest of the read-only gating) was clean or already covered. Submitting as comments, not a formal approval.

[David Pine (IEvangelist)]

Tested this locally against the Stress and TestShop playgrounds — the SQLite persistence modes (None/Run/Resume) and the header run-history selector all work as expected, including completed runs persisting across AppHost restarts and opening read-only. Approving.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

[James Newton-King (JamesNK)]

Superseded by the same-repository native stack #18926:

• Parent replacement: Add SQLite persistence and run history to the Dashboard #18924
• Pinning child: Add pinning to Dashboard run history #18925

Recreated upstream because GitHub native stacks require every branch to belong to the same repository.