feat(versioning): cross-entity version activity view

[mikebridge]

SUMMARY

Adds a backend-only cross-entity version activity view for dashboards, charts, and datasets:

GET /api/v1/{chart,dashboard,dataset}/<uuid>/activity/

The endpoints return a unified, newest-first stream of atomic version changes. Dashboard activity includes its own edits, edits to charts during their dashboard-association windows, and edits to datasets during the relevant chart/dataset windows. Chart activity includes its own edits plus dataset edits during association. Dataset activity is self-only.

The implementation reads the versioning infrastructure already present on master: dashboard_slices_version, slices_version, version_changes, and version_transaction. This PR contains no database migration.

Related records are filtered through the caller's read visibility. Historical identity uses stable UUIDs in addition to internal integer IDs so ID reuse cannot authorize predecessor history. Hard-deleted or UUID-mismatched related entities appear only as redacted tombstones: entity_deleted=true, entity_uuid=null, version_uuid=null, empty entity_name, generic (deleted) <kind> summary, and no editor/path/diff values.

API contract

Query parameters:

Parameter	Type	Default	Behavior
since	ISO 8601 date-time	—	Inclusive lower issued_at bound
until	ISO 8601 date-time	—	Exclusive upper issued_at bound
include	all | self | related	all	For datasets, related is empty and all is equivalent to self
q	string, maximum 1024 characters	—	Case-insensitive full-history search before pagination
page	integer ≥ 0	0	Offset page
page_size	integer ≥ 1	25	Silently clamped to 200

Response:

{
  "result": ["/* ActivityRecordSchema[] */"],
  "count": 0,
  "truncated": false
}

count covers the materialized filtered stream and is a floor when truncated=true. The fetch ceiling is applied across exact association windows after globally merging entity-ID chunks. truncated=true means an additional older exact-window match was observed beyond the ceiling.

Review remediation is tracked in specs/sc-107283-versioning-activity-view/spec.md (AV-021–AV-035) and tasks.md (T054–T068).

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Not applicable. This PR is backend-only; the consuming UI is tracked separately in #41551.

TESTING INSTRUCTIONS

source ~/venv/superset/bin/activate

pytest -q tests/unit_tests/versioning/test_activity.py
pytest -q \
  tests/integration_tests/versioning/activity_view_tests.py \
  tests/integration_tests/versioning/change_records_tests.py \
  tests/integration_tests/versioning/perf_validation_tests.py

pre-commit run --all-files

Manual smoke test after generating version history for a dashboard and its related charts/datasets:

curl "http://localhost:8088/api/v1/dashboard/$DASHBOARD_UUID/activity/" | jq
curl "http://localhost:8088/api/v1/dashboard/$DASHBOARD_UUID/activity/?include=self&page_size=10" | jq
curl "http://localhost:8088/api/v1/chart/$CHART_UUID/activity/?q=country" | jq
curl "http://localhost:8088/api/v1/dataset/$DATASET_UUID/activity/?include=related" | jq

The last request should return an empty result because datasets have no related layer in this version.

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

Prerequisites already merged: #39859 and #41176.

[netlify]

✅ Deploy Preview for superset-docs-preview ready!

Name	Link
🔨 Latest commit	8b19744
🔍 Latest deploy log	https://app.netlify.com/projects/superset-docs-preview/deploys/6a5573a859144e0008331344
😎 Deploy Preview	https://deploy-preview-41076--superset-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[codecov]

Codecov Report

❌ Patch coverage is 79.14184% with 175 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.12%. Comparing base (993a433) to head (ea43cd3).

Files with missing lines	Patch %	Lines
superset/versioning/activity/queries.py	78.05%	39 Missing and 22 partials ⚠️
superset/versioning/activity/impact.py	29.78%	31 Missing and 2 partials ⚠️
superset/versioning/activity/orchestrator.py	83.21%	16 Missing and 8 partials ⚠️
superset/versioning/activity/render.py	83.14%	11 Missing and 4 partials ⚠️
superset/versioning/activity/kinds.py	81.48%	6 Missing and 4 partials ⚠️
superset/versioning/activity/visibility.py	79.54%	7 Missing and 2 partials ⚠️
superset/versioning/changes/listener.py	20.00%	8 Missing ⚠️
superset/versioning/activity/windows.py	76.66%	4 Missing and 3 partials ⚠️
superset/versioning/api_helpers.py	82.60%	3 Missing and 1 partial ⚠️
superset/versioning/activity/scope.py	95.55%	0 Missing and 2 partials ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #41076      +/-   ##
==========================================
+ Coverage   65.05%   65.12%   +0.07%     
==========================================
  Files        2747     2756       +9     
  Lines      153766   154583     +817     
  Branches    35250    35395     +145     
==========================================
+ Hits       100027   100675     +648     
- Misses      51829    51952     +123     
- Partials     1910     1956      +46     

Flag	Coverage Δ
hive	38.71% <7.74%> (-0.34%)	⬇️
mysql	58.12% <79.14%> (+0.23%)	⬆️
postgres	58.17% <79.14%> (+0.22%)	⬆️
presto	40.68% <7.74%> (-0.37%)	⬇️
python	59.54% <79.14%> (+0.21%)	⬆️
sqlite	57.79% <79.14%> (+0.23%)	⬆️
unit	100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[mikebridge]

Versioning base-infra — verified review notes (head bf31b236)

Generated by a multi-lens review (committer / python / sqlalchemy / DDD lenses) and then independently verified against the source — every finding below was re-checked by reading the code at this head, and a few of the raw findings were dropped or down-scoped in that pass (noted where relevant). Line numbers are against bf31b236. SIP/process items are tracked separately and excluded here.

TL;DR — nothing here blocks the dark launch. It ships inert behind ENABLE_VERSIONING_CAPTURE (default off), the migration is additive/reversible, and the security posture (per-object raise_for_access on every read endpoint, fail-closed visibility filter, fail-open write path) is solid. The items below are (A) capture-path correctness bugs to fix before flipping the flag on, (B) small cleanups, and (C) forward-looking notes.

A. Fix before enabling ENABLE_VERSIONING_CAPTURE

These are latent while capture is off, so they don't block merge — but they bite the moment capture is enabled.

superset/versioning/diff.py:425-430 — duplicate natural keys silently overwrite, dropping change records

The by-key maps are built last-write-wins: two adhoc filters on the same subject column (or two metrics sharing a label) collapse to one, so editing/removing one emits a mangled single record and the other's change is lost. The _filter_key docstring's promise that same-column filters stay disambiguable can't hold once the collision drops one.

Would it be worth disambiguating on collision (secondary key, or fall back to positional index only for colliding items)?

superset/versioning/diff.py:420-427 (+ listener catch at changes/listener.py:239) — an unhashable subject raises TypeError, losing that entity's scalar diff

_filter_key returns f.get("subject") (typed Key, but it's Any from user JSON). A list/dict subject flows unguarded into from_by_key[<list>] → TypeError: unhashable. It's caught per-entity, so the blast radius is just that one entity's scalar change records for the save (not the whole save) — but the record loss is silent. A str(...) coercion (or a fallback in _effective_key when the key isn't str/int/None) closes it.

superset/commands/importers/v1/utils.py:391-419 — _prime_versioning_unit_of_work's broad except gives false assurance

The docstring promises it "never breaks an import," but the broad except Exception only genuinely covers the ImportError (continuum-absent) case. If unit_of_work() fails for any other reason while capture is enabled, the very next dashboard_slices.insert() fires Continuum's before_execute with no UoW and raises KeyError — the import dies anyway, now with a misleading "proceeding without it" log. Narrowing to except ImportError: lets the real failure surface at its source. (Narrow trigger — unit_of_work rarely fails — but the false assurance is the issue.)

superset/versioning/activity/queries.py:444-489 — per-kind budget can drop later chunks' records and mislabel the window as complete

When one kind's entities span multiple id-chunks and fill the ~5000-row budget, the loop breaks and later chunks contribute zero rows — including records newer than the truncation floor — yet truncated=True presents it as a clean "older records omitted" cut. (Correction to the raw finding: the drop order is arbitrary but deterministic — not PYTHONHASHSEED-dependent — since these are set[int]. High bar: >500 entities of one kind and >5000 matching rows.) Applying the ceiling after concatenating + re-sorting all chunks (and sorted(entity_ids) before chunking) makes any residual drop deterministic and correctly labeled.

B. Small cleanups (unconditional)

superset/versioning/diff.py:799-805 — _meta_excluding_position doesn't exclude anything

Name + docstring promise positional bits are stripped from meta; the body just returns dict(meta) unchanged. It's correct-by-accident today (position lives on node["parents"], handled separately), but the next field added to meta silently breaks the move-vs-edit contract the name advertises. Rename to _meta_copy (+ honest docstring) or actually implement the strip.

Slice → chart dispatch is spread across ~8 parallel structures keyed on the class-name string

ENTITY_KIND_BY_CLASS_NAME (changes/table.py:86), TABLE_KIND_TO_API / API_KIND_TO_TABLE / API_KIND_LABEL / USER_FACING_KIND / NOT_FOUND_EXC / NAME_COLUMN (activity/kinds.py:73-115), _RAISE_FOR_ACCESS_KWARG (api_helpers.py:136), plus string-dispatch ladders in changes/state.py. Adding a fourth versioned entity means editing all of them in lock-step with nothing catching a miss. A single registry (one entry per entity, constructor-checked for completeness) turns "did I update every table?" into a type error. The DDD lens flagged this as the one durable must-fix as the surface grows.

superset/daos/dataset.py:534-537 & :602-605 — _upsert_columns / update_metrics lack the protected_keys guard

The sibling _override_columns (:512-516) protects ("id", "table_id") before setattr; these two loops set every payload key unguarded. Benign today, but it's the exact footgun the sibling was hardened against — worth the same guard for symmetry.

C. Forward-looking (not this PR, but worth a note)

• Shadow-table FKs to version_transaction have no ondelete, while version_changes.transaction_id has ondelete="CASCADE" (migration 56cd24c07170). When the retention task (stacked PR) deletes version_transaction rows, it must delete referencing shadow rows first, in order, or every prune cycle hits an FK violation on PG/MySQL. Either add ondelete="CASCADE" to the six shadow FKs, or make the deletion-ordering contract explicit where retention lands.
• Migration idempotency is inconsistent — add_versioning_tables uses raw create_table (a partial failure on MySQL's non-transactional DDL isn't re-runnable), while the index follow-up migration is inspector-guarded. Worth matching the guarded pattern (or documenting the manual-cleanup caveat).
• Per-save cost once capture is on — SkipUnmodifiedPlugin._matches_previous_version (factory.py:294) + _read_pre_state add two full-row reads (incl. large MediumText columns) per versioned save. perf_validation_tests.py is the right place to surface numbers before the default flips.

Checked and cleared

• SqlaTable.__versioned__ excluding editors but not viewers — not a problem: SqlaTable has no viewers relationship, so there's nothing to exclude (unlike Dashboard/Slice).
• Import-time make_versioned() in superset/extensions/__init__.py — no model classes are imported at that point, so it's the correct Continuum ordering, not an import-safety regression.

Praise where due: the fail-open write path + dedicated capture-error metrics, the thorough kill-switch (detaches Continuum's own listeners and verifies), the deterministic UUIDv5 version identity, the pure/functional diff engine with ChangeRecord/Window as clean value objects, and an unusually honest UPDATING.md are all genuinely well done.

[mikebridge]

🤖 This review was generated via Codex using parallel Preset, Python, React, and Superset committer review passes.

Thanks for the substantial work on the cross-entity activity view. The implementation and test coverage are extensive, but I found two correctness issues that should be resolved before merging.

Blocking findings

1. The per-kind cap can omit newer activity across ID chunks

In superset/versioning/activity/queries.py:445, each ID chunk is queried using only the budget left by earlier chunks. Because the IDs originate from sets, chunk order is not a meaningful global ordering.

An earlier chunk can consume the 5,000-row cap even when a later chunk contains newer activity. The resulting stream can therefore return older records while silently omitting newer ones.

Could we instead fetch a bounded candidate set from every chunk, merge the candidates using the complete ordering key, and retain the newest global limit rows? Fetching or probing one additional row would also let us set truncated only when more data actually exists.

A regression test with more than 500 related entities, where a later chunk contains the newest record, would help lock this down.

2. Related history is not bounded to the current entity incarnation

In superset/versioning/activity/scope.py:73, self_start_tx protects records for the entity itself, but related traversal can still read relationship and entity-version rows associated with a previously deleted object that used the same integer ID.

If the database reuses an ID, the newly created dashboard or chart could display activity belonging to its predecessor.

Could we propagate the lower transaction boundary for the current entity into related-scope resolution and clip the relationship windows accordingly? Matching shadow records against the current UUID where available would provide additional protection.

Minor documentation points

• PathEntityResponseError and resolve_endpoint_path_entity in superset/versioning/api_helpers.py:143 should begin with complete, single-line PEP 257 summaries.
• The dated profiling reference in superset/versioning/activity/queries.py:196 should use timeless wording while preserving the useful performance rationale.

Overall, the architecture looks thoughtfully developed, but the chunk-ordering and entity-incarnation issues can affect the correctness of the activity stream and should be addressed before merge.

[mikebridge]

Review generated by Codex.

The activity stream can disclose historical data under integer-ID reuse and can omit valid records when its fetch ceiling interacts with broad windows or ID chunking. Its OpenAPI response contract is also incomplete and inconsistent with tombstone responses.

P1 — Treat reused related IDs as tombstones

superset/versioning/activity/visibility.py:201-205

When a related chart or dataset is hard-deleted and its integer ID is later reused, this checks historical records against the replacement live row because identity is keyed only by (kind, id). A Gamma principal who can read the path dashboard and replacement dataset can then receive the old dataset's unredacted name, editor, path, and values despite not being granted that dataset, violating the SECURITY.md Gamma row (only granted datasets). Please compare the historical shadow UUID with the live UUID and treat mismatches as tombstones.

P2 — Apply the fetch limit after exact window filtering

superset/versioning/activity/queries.py:460-463

When one entity has many recent changes outside its own association window, the kind-wide transaction bounds still admit those rows, allowing them to consume the 5,000-row SQL limit before the Python window filter discards them. For example, a heavily edited chart after detachment can crowd out valid older activity from attached charts, yielding a silently incomplete stream. Please apply the limit across exact per-entity windows or continue fetching after discarded rows.

P2 — Merge chunk results before enforcing the per-kind limit

superset/versioning/activity/queries.py:445-448

When a kind spans more than 500 entity IDs, chunks are queried sequentially and the first chunks consume the shared limit without comparison to later chunks. If an early chunk supplies 5,000 rows, newer records from every later chunk are omitted, and the reported truncation floor does not make the returned interval complete. Please fetch per-chunk candidates and globally merge/order them before applying the limit.

P2 — Register the activity response schema

superset/charts/api.py:1593

The new endpoint references ActivityResponseSchema, but none of the chart, dashboard, or dataset APIs imports it or adds it to openapi_spec_component_schemas (unlike VersionListItemSchema). Consequently, the generated OpenAPI document cannot resolve this response schema. Please register it for all three resource APIs.

P2 — Mark tombstone version UUIDs as nullable

superset/versioning/schemas.py:275-284

For hard-deleted related entities, decoration deliberately emits version_uuid=None because no live UUID is available, but this field is declared as a non-null string. Once the response schema is exposed in OpenAPI, generated clients will reject a documented tombstone response. Please set allow_none=True and document the tombstone case.

[mikebridge]

Follow-up on the review findings posted above: the blocking correctness items and associated cleanup were addressed in 0d6175e5bd, 9a855a71c0, and ea43cd30a2.

In particular, the updated implementation:

• globally merges bounded candidates across entity-ID chunks, preserving the newest records regardless of chunk order and reporting truncation consistently;
• bounds self and related history to the current (id, uuid) incarnation, preventing history leakage when integer IDs are reused;
• resolves historical UUIDs during decoration and redacts records that cannot be matched safely;
• handles degenerate shadow rows without turning an activity request into a 500;
• bounds name/impact lookups to the page transaction range and includes the smaller documentation/docstring cleanups;
• adds regression coverage for later-chunk newer records, current-incarnation clipping, empty untracked related scope, reused-ID redaction, and stable total ordering.

I also performed a fresh Docker/PostgreSQL dark-launch verification with ENABLE_VERSIONING_CAPTURE absent from the environment (resolved by Superset as False). I committed description changes to an existing chart, dashboard, and dataset and confirmed that the saves persisted while all capture tables remained empty:

• slices_version: 0
• dashboards_version: 0
• tables_version: 0
• version_transaction: 0
• version_changes: 0

Authenticated requests to each entity's /activity/ endpoint also returned count=0, an empty result, and truncated=false. This confirms the feature remains inactive by default while the read endpoints continue to behave normally.

[rusackas]

Thanks @mikebridge, this is a lot of careful work, and the review threads look properly resolved rather than just closed. LGTM.

One loose end left: the schema-registration and allow_none items from your own July 13 pass (ActivityResponseSchema missing from openapi_spec_component_schemas, version_uuid still non-nullable) don't look like they made it into the fixes. Worth a follow-up commit, but not holding up the merge for it.