Skip to content

feat(ingest/superset): emit dataPlatformInstance for dashboards and charts - #19879

Open
Danishlynx wants to merge 1 commit into
datahub-project:masterfrom
Danishlynx:feat-superset-platform-instance
Open

Danishlynx wants to merge 1 commit into
datahub-project:masterfrom
Danishlynx:feat-superset-platform-instance

Conversation

@Danishlynx

@Danishlynx Danishlynx commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #11153.

Superset has accepted platform_instance since #9647 but never emitted the dataPlatformInstance aspect, so the Navigate panel files every dashboard and chart under "Default" no matter how many deployments you ingest. The code carried a TODO for exactly this.

Changes

  • superset.py: a get_data_platform_instance helper, modelled on Tableau's except that it returns nothing when platform_instance is unset, so existing deployments and the five existing goldens are untouched; the aspect appended at the dashboard and chart snapshot sites; the PLATFORM_INSTANCE capability declared. Preset builds on this source and inherits both with no code of its own; a unit test pins that its aspect resolves to urn:li:dataPlatform:preset.
  • Tests: unit present/absent cases for dashboards and charts, an empty-string case, the Preset inheritance and capability cases, and one integration test with a new golden that sets platform_instance.
  • Docs: a Platform instance section on both connector pages.
  • The connector registry JSON regenerated (it gains the capability rows for superset and preset).

Datasets are deliberately not in this PR

Superset builds dataset URNs through get_datasource_urn_from_id, which uses plain make_dataset_urn, so those URNs carry no instance (the instance-aware gen_dataset_urn helper exists in the file but is unused). Emitting the aspect on datasets would name an instance the URN does not have, and two deployments exposing the same database, schema and table would overwrite each other's aspect on the one entity. Making the dataset URN instance-aware is a breaking URN change that also has to move the chart-to-dataset lineage edges, so it belongs with the containers follow-up rather than here. A unit test pins that datasets get no aspect, the TODO says why, and the docs say the same.

Notes for review

  • Snapshot style kept: the aspect joins the existing DashboardSnapshot and ChartSnapshot aspect lists rather than going out as a separate MCP, matching how this source emits today. Containers in the follow-up will go out as MCP workunits beside the snapshots, the way Mode does.
  • Goldens: the five pre-existing superset and preset goldens are byte-identical, which is the evidence that nothing changes when platform_instance is unset. The new golden carries exactly seven aspects (two dashboards, five charts).
  • Follow-ups, in order: database and schema containers, the dataset instance question above, and test_connection.

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues
  • Tests for the changes have been added/updated
  • Docs related to the changes have been added/updated

Summary by cubic

Fixes #11153. Superset has accepted platform_instance since #9647 but never emitted the dataPlatformInstance aspect, so the Navigate panel filed every dashboard and chart under "Default". Now Superset and Preset emit the aspect on dashboards and charts when platform_instance is set; when it isn't, output is unchanged.

Datasets

  • Datasets are excluded because their URNs don't carry an instance.
  • Making dataset URNs instance-aware is a breaking change and will follow with the containers work.

Tests and docs

  • Adds unit coverage for present, absent, empty-string, and Preset-inheritance cases plus one integration golden.
  • Adds a Platform instance section to both connector docs and regenerates the connector registry with the PLATFORM_INSTANCE capability.

Written for commit 637466a. Summary will update on new commits.

Review in cubic

@github-actions github-actions Bot added ingestion PR or Issue related to the ingestion of metadata community-contribution PR or Issue raised by member(s) of DataHub Community labels Sep 20, 2026
@maggiehays maggiehays added the needs-review Label for PRs that need review from a maintainer. label Sep 20, 2026
…harts

Superset has accepted platform_instance since datahub-project#9647 but never emitted
the dataPlatformInstance aspect, so the Navigate panel files every
dashboard and chart under "Default" no matter how many deployments are
ingested.

Add a get_data_platform_instance helper, modelled on Tableau's except
that it returns nothing when platform_instance is unset, so existing
deployments and the five existing goldens are untouched. Append the
aspect at the dashboard and chart snapshot sites and declare the
PLATFORM_INSTANCE capability. Preset builds on this source and inherits
both with no code of its own; a unit test pins that its aspect resolves
to the preset platform.

Datasets are deliberately left out. Their URNs are built through
get_datasource_urn_from_id with plain make_dataset_urn and carry no
instance (the instance-aware gen_dataset_urn helper exists but is
unused), so an aspect on them would name an instance the URN does not
have, and two deployments exposing the same table would overwrite each
other's aspect on the one entity. Making the dataset URN instance-aware
is a breaking change that also moves chart-to-dataset lineage, so it
belongs with the containers follow-up. A unit test pins the omission and
the TODO says why.

Tests: unit present/absent cases for dashboards and charts, an
empty-string case, the Preset inheritance cases, and one integration
test with a new golden that sets platform_instance. Docs: a Platform
instance section on both connector pages. The connector registry JSON
is regenerated.

Fixes datahub-project#11153

This branch was successfully deployed

1 active deployment
Preview — 637466a1 Deployed Sep 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution PR or Issue raised by member(s) of DataHub Community ingestion PR or Issue related to the ingestion of metadata needs-review Label for PRs that need review from a maintainer.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

platform_instance is not shown for Superset platform assets

2 participants