Use SQLite for dashboard telemetry filtering

[James Newton-King (JamesNK)]

Description

Dashboard telemetry filtering was implemented in both the production SQLite repository and the in-memory repository used by tests. This allowed test behavior and UI filter checks to diverge from the queries used by the running dashboard.

This change makes SQLite the single owner of telemetry filtering behavior. It removes InMemoryTelemetryRepository, updates tests to create and dispose temporary SQLite repositories, and removes the in-memory TelemetryFilter.Apply implementations. Watchers and selected-item checks now constrain their SQL queries by telemetry identity, while trace detail filtering retains its tree expansion behavior after SQLite identifies direct matches.

Span display summaries are persisted so waterfall text searches remain simple LIKE expressions. The change also corrects negative service-name filtering and prevents stale asynchronous filter results from clearing a newer structured-log selection.

User-facing usage

The existing dashboard trace and structured-log search and field-filter controls continue to work as before, but now use the same SQLite query semantics throughout. Trace searches continue to match rendered summaries such as HTTP GET 200, and negative service-name filters apply to both source and populated peer services.

Validation

• Aspire.Dashboard.Tests: 1,726 passed
• Aspire.Dashboard.Components.Tests: 258 passed
• git diff --check

Fixes # (issue)

Checklist

• Is this feature complete?
  • Yes. Ready to ship.
  • No. Follow-up changes expected.
• Are you including unit tests for the changes and scenario tests if relevant?
  • Yes
  • No
• Did you add public API?
  • Yes
    • If yes, did you have an API Review for it?
      • Yes
      • No
    • Did you add <remarks /> and <code /> elements on your triple slash comments?
      • Yes
      • No
  • No
• Does the change make any security assumptions or guarantees?
  • Yes
    • If yes, have you done a threat model and had a security review?
      • Yes
      • No
  • No

[github-actions]

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 19661

Or

• Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 19661"

[Copilot]

Pull request overview

Moves dashboard telemetry filtering to SQLite so tests and production share query semantics.

Changes:

• Removes in-memory filtering and test repositories.
• Adds SQL-backed span/log filtering, watcher matching, and persisted span summaries.
• Updates trace and structured-log UI filtering with stale-result protection.

Reviewed changes

Copilot reviewed 32 out of 33 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/Shared/Telemetry/TelemetryTestHelpers.cs	Removes in-memory repository factory.
tests/Shared/Telemetry/InMemoryTelemetryRepository.Watchers.cs	Deletes in-memory watcher implementation.
tests/Aspire.Dashboard.Tests/TelemetryRepositoryTests/TraceTests.cs	Adds SQLite filtering coverage.
tests/Aspire.Dashboard.Tests/TelemetryRepositoryTests/TelemetryRepositoryTests.cs	Removes in-memory test variant.
tests/Aspire.Dashboard.Tests/TelemetryRepositoryTests/TelemetryRepositoryTestBase.cs	Always creates temporary SQLite repositories.
tests/Aspire.Dashboard.Tests/TelemetryRepositoryTests/TelemetryLimitTests.cs	Removes in-memory test variant.
tests/Aspire.Dashboard.Tests/TelemetryRepositoryTests/ResourceTests.cs	Removes in-memory test variant.
tests/Aspire.Dashboard.Tests/TelemetryRepositoryTests/MetricsTests.cs	Removes in-memory test variant.
tests/Aspire.Dashboard.Tests/TelemetryRepositoryTests/LogTests.cs	Removes in-memory test variant.
tests/Aspire.Dashboard.Tests/TelemetryApiServiceTests.cs	Migrates API tests to SQLite.
tests/Aspire.Dashboard.Tests/Shared/SqliteRepositoryTestHelpers.cs	Adds temporary SQLite test contexts.
tests/Aspire.Dashboard.Tests/Model/TelemetryImportServiceTests.cs	Migrates import tests to SQLite.
tests/Aspire.Dashboard.Tests/Model/StructuredLogsPageViewModelTests.cs	Tests SQL-backed selection filtering.
tests/Aspire.Dashboard.Tests/Model/SpanWaterfallViewModelTests.cs	Removes in-memory filter tests.
tests/Aspire.Dashboard.Tests/Model/ResourceMenuBuilderTests.cs	Migrates menu tests to SQLite.
tests/Aspire.Dashboard.Tests/Model/GenAIVisualizerDialogViewModelTests.cs	Migrates GenAI tests to SQLite.
tests/Aspire.Dashboard.Components.Tests/Shared/TestTelemetryRepository.cs	Adds injectable log-query handling.
tests/Aspire.Dashboard.Components.Tests/Pages/TraceDetailsTests.cs	Tests SQL match-based waterfall filtering.
tests/Aspire.Dashboard.Components.Tests/Pages/StructuredLogsTests.cs	Tests stale filter-query protection.
src/Aspire.Dashboard/ServiceClient/DatabaseSchema/004.Traces.sql	Stores span display summaries.
src/Aspire.Dashboard/ServiceClient/DashboardSqliteDatabase.cs	Increments schema version.
src/Aspire.Dashboard/Otlp/Storage/SqliteTelemetryRepository.Traces.Writes.cs	Persists display summaries.
src/Aspire.Dashboard/Otlp/Storage/SqliteTelemetryRepository.Traces.Reads.cs	Adds SQL span filtering and identity constraints.
src/Aspire.Dashboard/Otlp/Storage/SqliteTelemetryRepository.Runtime.cs	Filters watcher pushes through SQLite.
src/Aspire.Dashboard/Otlp/Storage/SqliteTelemetryRepository.Logs.cs	Adds log-ID-constrained filtering.
src/Aspire.Dashboard/Otlp/Storage/GetSpansRequest.cs	Adds internal span identity constraints.
src/Aspire.Dashboard/Otlp/Storage/GetLogsContext.cs	Adds internal log-ID constraints.
src/Aspire.Dashboard/Model/SpanType.cs	Removes in-memory filter evaluation.
src/Aspire.Dashboard/Model/Otlp/TelemetryFilter.cs	Removes model-side filter evaluation.
src/Aspire.Dashboard/Model/Otlp/SpanWaterfallViewModel.cs	Removes local waterfall matching.
src/Aspire.Dashboard/Components/Pages/TraceDetail.razor.cs	Queries SQLite for matching span IDs.
src/Aspire.Dashboard/Components/Pages/StructuredLogs.razor.cs	Queries SQLite before clearing selections.

[Copilot]

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated no new comments.

Suppressed comments (2)

src/Aspire.Dashboard/Components/Pages/StructuredLogs.razor.cs:400

• Capture the selected log identity before the query and re-check it before clearing. If log A is selected when this query starts and the user selects log B while it awaits SQLite, an excluded result for A currently clears B. This directly reintroduces the stale asynchronous selection race the PR description says is prevented.

        var isExcluded = await PageViewModel.IsSelectedLogEntryExcludedByFiltersAsync(
            TelemetryRepository,
            textFilter,
            fieldFilters,
            _cts.Token);
        if (isExcluded)

src/Aspire.Dashboard/Components/Pages/TraceDetail.razor.cs:632

• Guard these asynchronous results with a per-filter request version before assigning them. Blazor can process another filter event while this SQLite query is awaiting; if the newer query completes first, the older continuation then overwrites both match sets and ClearSelectedDataIfNotVisible can clear the selection using stale filters. The existing detail-update version does not cover filter-only changes.

        // An older filter query could finish after a newer query and apply stale matches. Each query is
        // constrained to the current trace and filter changes are user-driven, so this is unlikely and not
        // worth the additional state and coordination required to guard against it.

[Copilot]

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated no new comments.

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

src/Aspire.Dashboard/Components/Pages/StructuredLogs.razor.cs:395

• Overlapping filter callbacks can complete out of order here. An older restrictive query can return after a newer query (or after the user selects another log) and ClearSelectedLogEntryAsync then clears the current selection using stale state—the race this PR is intended to prevent. Assign each filter check a monotonically increasing version, capture the selected entry before awaiting, and clear only when both are still current; add a delayed-repository regression test.

        // An older filter query could finish after a newer query and clear the selection using stale filters.
        // The query is constrained to one log and filter changes are user-driven, so this is unlikely and not
        // worth the additional state and coordination required to guard against it.
        var isExcluded = await PageViewModel.IsSelectedLogEntryExcludedByFiltersAsync(

src/Aspire.Dashboard/Components/Pages/TraceDetail.razor.cs:658

• Delegating text matching entirely to SQLite drops matching against the resource labels rendered by this page. The removed path used OtlpHelpers.GetResourceName(..., _resources), which disambiguates replicas as service-<instance>; SQLite only has separate resource_name/instance_id columns and therefore cannot match the displayed composite source or peer label. Searching the visible replica name now hides that span. Include those display aliases in repository matching or preserve an alias-aware page-side match, with replica coverage.

                Count = int.MaxValue,
                Filters = filters,
                TraceId = traceId,
                TextFragments = textFragments
            }, _cts.Token);

src/Aspire.Dashboard/Components/Pages/TraceDetail.razor.cs:634

• These two repository calls can overlap across rapid text/type/dialog changes, so an older result can overwrite ContextFilterMatches/DurationFilterMatches after the latest result and RefreshAfterFilterChangeAsync may then clear a valid selection. The same race also spans route updates because the component cancellation token is not renewed when TraceId changes. Guard assignments with a dedicated filter-query version (and the captured trace ID), then test out-of-order completions.

        // An older filter query could finish after a newer query and apply stale matches. Each query is
        // constrained to the current trace and filter changes are user-driven, so this is unlikely and not
        // worth the additional state and coordination required to guard against it.
        var hasTextFilter = !string.IsNullOrWhiteSpace(PageViewModel.Filter);
        var contextMatches = contextFilters.Count > 0 || hasTextFilter

[github-actions]

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

[Mitch Denny (mitchdenny)]

James Newton-King (@JamesNK) had a look at this PR, test coverage looks solid on the new backend.

[github-actions]

🔍 CI Failure Analysis: Transient Infrastructure Failure

The CI build failed due to transient infrastructure issues.

Failed jobs:

• Tests / RabbitMQ.Client / RabbitMQ.Client (windows-latest) — All failed steps show 'Process completed with exit code -1073741502' (0xC0000142 / STATUS_DLL_INIT_FAILED), a known transient Windows runner process-initialization failure occurring in post-test steps (hang dump check, upload logs/results, cleanup). No test failures were extracted from TRX artifacts, and the failure is unrelated to the RabbitMQ.Client test area or any PR-changed files (which are all in Aspire.Dashboard). (transient-infra)

If a rerun was not already requested automatically, visit the workflow run page to rerun the failed jobs manually.