chore(viz): remove legacy explore_json + viz.py pipeline

[rusackas]

SUMMARY

This branch removes the legacy viz pipeline — the deprecated /superset/explore_json/ endpoints and superset/viz.py (both @deprecated(eol_version="5.0.0") since 3.0) — by first migrating all 15 remaining useLegacyApi charts to /api/v1/chart/data, then deleting the pipeline and dropping the legacy- prefix from the surviving plugin packages.

How the charts were migrated. Eleven charts were converted in place: same renderer, same viz_type (so saved charts keep working with no DB migration), with a new buildQuery.ts mirroring the old viz.py query_obj() and the get_data() reshape ported to transformProps — each with Jest tests pinning parity against the legacy behavior (null padding, pandas sort orders, comma escaping, fill_value semantics, relativedelta/offset-rollback arithmetic, etc.). Where the legacy engine used pandas transforms with post-processing equivalents (rose), the new buildQuery attaches the server-side post_processing pipeline (pivot → resample → rolling → compare → contribution → flatten) instead of re-implementing pandas in JS. Two charts were retargeted to existing ECharts charts via MigrateViz processors + Alembic revisions with reversible form_data backups: bubble → bubble_v2 (the processor existed since the ECharts bubble shipped, but no revision ever ran it) and compare → echarts_timeseries_line. deck_multi became fully client-side, completing the deck.gl preset migration: the container issues an empty-queries context and each layer loads through its own registered buildQuery/transformProps, resolving its datasource from the chart API's authoritative datasource_id/type rather than the (possibly stale) params.datasource string, and surfacing per-layer load failures as a warning banner instead of failing silently.

Beyond the original scope: nvd3 is now completely gone. Three of the migrated charts didn't stop at a new data pipeline — they were subsequently rebuilt as native ECharts renderers, which meant the entire nvd3/nvd3-fork dependency (and the last package that depended on it, preset-chart-nvd3) could be deleted from the codebase outright:

• Nightingale Rose — polar stacked bars with a sqrt-normalized area-proportional mode, plus a click-to-drill morph into a single-period pie via ECharts universalTransition.
• Bullet Chart — nested markArea range bands, scatter/triangle target markers, and per-range tooltips.
• Time-series Period Pivot — one line per period, current period bold, priors faded.
• As part of this, compare's percent-change re-basing (originally dropped as a "known limitation" of the ECharts migration — see below) was restored on the mainline echarts_timeseries_line chart via a rebase_percent_change flag and a draggable baseline (ECharts graphic overlay + convertFromPixel + grid-rect-clamped snapping).
• paired_ttest's table was also rebuilt on antd Table, dropping the unmaintained reactable dependency (which turned out to have a real production-only rendering bug — it silently drops rows in a minified bundle, invisible in Jest — so this was a correctness fix, not just cleanup) and the Node-only distributions package.

What was removed (#41750): the explore_json routes and their CSV/XLSX/query/results/samples helpers, superset/viz.py and every importer (get_viz, Slice.viz, Slice.explore_json_url, legacy cache warm-up, legacy annotation data, the BaseViz overload of raise_for_access), the load_explore_json_into_cache celery task, the can_explore_json public-role grant, the CSRF exemption, and the frontend useLegacyApi plumbing (ChartMetadata field, shouldUseLegacyApi, all /explore_json/ request branches, and a dataset-required warning banner that only ever applied to legacy chart types).

Renames (#41751): legacy-plugin-chart-{calendar,chord,country-map,horizon,paired-t-test,parallel-coordinates,partition,rose,world-map} → plugin-chart-*; legacy-preset-chart-nvd3 → preset-chart-nvd3, which itself was later deleted outright once Bullet, Time-series Period Pivot, and Rose no longer needed it. "(legacy)" chart names and Legacy gallery tags are gone.

Phase PRs (each merged green with bot-review threads resolved): #41715 (dead code), #41716 para, #41717 country_map, #41718 bullet (data-pipeline migration; later superseded by the ECharts rebuild below), #41719 chord, #41720 world_map, #41721 paired_ttest, #41723 time_table, #41724 cal_heatmap, #41725 horizon, #41726 rose (data-pipeline migration; later superseded), #41727 time_pivot (data-pipeline migration; later superseded), #41728 bubble, #41729 partition, #41730 deck_multi, #41732 (ordering parity), #41738 compare, #41750 (pipeline removal), #41751 (renames), #42225 (Bullet rebuilt on ECharts), #42245 (Time-series Period Pivot rebuilt on ECharts), #42247 (percent-change rebasing restored), #42381 (Rose rebuilt on ECharts, preset-chart-nvd3 package deleted), #42530 (post-review hardening from a self-review pass over this branch — see below).

Known behavior notes (also in UPDATING.md): deck_multi dashboard filter badges no longer aggregate child-layer filter metadata (each layer is now an independently-fetched chart, so there's no single payload to summarize from); the world bank example ships bubble_v2. The two previously-listed limitations — compare's percent re-basing and deck_multi's initial autozoom — have since been resolved (see above and the deck_multi autozoom fix that refits the viewport as each layer's async features arrive, rather than only once at load).

Embedded/guest chart access — checked, mitigated. Charts on the v1 /api/v1/chart/data pipeline are subject to the guest-tamper check (query_context_modified) when embedded, which used to reject a request if the stored chart had no query_context and its stored params didn't look like the posted payload — explore_json was exempt, so legacy chart types embedded via guest tokens never hit it. Migrating 15 more chart types onto the v1 pipeline would have widened that exposure, but #42295 (already merged into this branch) added an equivalence table so the guest-tamper check compares against each chart type's own control-specific stored param keys (metric/x/y/size/entity/etc.) instead of requiring a stored query_context at all. Verified this is actually in the merged code (superset/security/manager.py, _columns_metrics_modified/_STORED_METRIC_PARAMS), not just present on master in isolation — flagging here so reviewers don't need to re-derive or re-flag it.

Migration downgrade scoping — fixed via #42530. MigrateViz.downgrade() selects rows by viz_type == target_viz_type AND params LIKE '%form_data_bak%', which isn't scoped to which migration produced them. Both compare→echarts_timeseries_line (this PR) and the already-shipped line→echarts_timeseries_line migration target the same viz_type, so running alembic downgrade on the compare migration (or superset viz-migrations downgrade -t compare) used to also revert any previously-migrated line charts back to the now-deleted legacy line plugin. #42530 (already merged into this branch) fixed this: downgrade_slice now checks each row's own form_data_bak["viz_type"] against source_viz_type before reverting it, so a downgrade only touches the slices the matching migration actually upgraded — the SQL-level filter stays coarse (it's still just target_viz_type-based, for efficient row selection), but the per-row check now makes it precise. The CLI's separate PREVIOUS_VERSION mapping, which had its own copy of this same source→target logic and could drift from it, was removed in favor of deriving the source viz type from each slice's own stored backup.

Further hardening from a self-review pass (#42530, already merged into this branch): a stale-response race in deck_multi (an out-of-order async layer response could clobber a newer one — fixed with a load-generation guard) plus removal of a dead code path reading a legacy payload shape the v1 endpoint never returns; the percent-change draggable baseline no longer reads its own series data back out of chart.getOption() post-setOption() (now reads the echartOptions prop directly) and gained its first test coverage; a query_context missing the expected "queries" key (e.g. hand-edited via the API) no longer throws an uncaught KeyError mid-upgrade and leaves the slice half-migrated; Bullet's and Time-series Period Pivot's near-duplicate wrapped-legend-row estimation logic was deduped into one shared, tested helper (which also fixed a legend/chart overlap bug in Time-series Period Pivot); partition, paired_ttest, and parallel_coordinates's near-duplicate sort-metric/orderby buildQuery logic was likewise deduped; and Rose, partition's transformProps, and the deck.gl Multi container all gained test coverage they were missing.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Converted-in-place charts render identically (same renderer, same data shape, different endpoint). Bubble/compare charts re-open as their ECharts equivalents. Bullet, Time-series Period Pivot, and Rose render as newly-built ECharts charts (see screenshots below) rather than their old nvd3 renderers.

TESTING

• ~90+ new Jest tests across the migrated plugins (buildQuery shapes, transformProps parity incl. legacy-payload pass-through), plus new suites for the three ECharts rebuilds (Bullet, Time-series Period Pivot, Rose, and the percent-change rebase math) and a backend unit test for the empty-queries query context; full unit and integration suites green on every phase PR and on the final merged state.
• DB migrations: superset db upgrade migrates saved bubble/compare charts (reversible via form_data_bak; superset viz-migrations CLI also supports both types).
• Manual: open any formerly-legacy chart — the network tab shows POST /api/v1/chart/data; /superset/explore_json/ no longer exists.
• 75 commits total, spanning 3 rounds of rebasing onto a fast-moving master (each caught and fixed a real Alembic multi-head conflict from parallel migration work landing on both sides).

Did manual tests of all affected charts... some are now better than they were on master. Many screenshots below.

SPECIAL BONUS: #42328 provides:

• New (not amazing, but I'm working on it) built-in example charts for ALL VISUALIZATIONS
• A new process to crawl and screencap ALL example charts as viz thumbnails

This can provide (a) evergreen thumbnails, and (b) regression testing for examples, a they should generally remain byte identical. In fact, I've been using that to find and fix bugs on this branch.

ADDITIONAL INFORMATION

• Has associated issue: Closes 89 - Migrate and remove the legacy Nightingale chart #41046, closes 83 - Make the Deck.GL plugin non-legacy #41047, closes 82 - Add CHART_PLUGINS_DEPRECATED feature flag #41048
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided: proportional to saved bubble/compare slice counts (paginated updates); no downtime expected
• Introduces new feature or API
• Removes existing feature or API

🤖 Generated with Claude Code

Screenshots from testing (see lots more on #42328)

New Nightengale Rose (ECharts)

Country Map:

World Map:

Chord:

Calendar Heatmap:

Horizon:

Bullet:

Timeseries Table

Nightengale Rose

Paired T-Test

After much ado, this one's working, and removes the reactable dependency to boot!

DeckGL Multi Layer

Timeseries pivot (not sure if this is working right... I never use this one. Investigating still...

Bubble

Partition

Parallel Coordinates

DeckGL Charts

[netlify]

✅ Deploy Preview for superset-docs-preview ready!

Name	Link
🔨 Latest commit	3929946
🔍 Latest deploy log	https://app.netlify.com/projects/superset-docs-preview/deploys/6a7427c807dafe00086dcda4
😎 Deploy Preview	https://deploy-preview-41714--superset-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[sancho11]

I've been running an adaptation of #41730 (the deck_multi migration merged into this branch) in production, on an instance with GLOBAL_ASYNC_QUERIES enabled and an embedded dashboard. It works, but I hit four issues that I think will show up for everyone once this lands. All four are in Multi.tsx, its metadata, none require changes to the removal itself.

Sharing them here with the fixes I ended up with, in case they're useful. Happy to open a PR against this branch for any or all of them.

Confidence, so you can weigh them: 2, 3 and 4 I watched fail in production and then traced to the code below. 1 I derived from the code and fixed before it could bite, so I have not actually witnessed it; treat it as a code reading rather than a report.

1. Autozoom silently stops working

getAdjustedViewport still reads the container's pre-merged features:

const features = props.payload?.data?.features || {};

But the new Multi/buildQuery.ts returns buildQueryContext(formData, () => []), so the container issues no query and that field is never populated. points ends up empty, fitViewport is never called, and the chart opens at the saved viewport instead of fitting the data. The // there may be none here comment in the diff acknowledges the empty case, but nothing replaces the points.

Fix: collect the points from the features each layer's own transformProps returned, aggregated by viz_type after the layers resolve. This is arguably better than the old behavior, it fits the data actually being rendered rather than a separately-computed merge.

2. The container renders the "No results" empty state

With queries: [] the endpoint answers result: [], and in SuperChart:

const noResultQueries =
  enableNoResults &&
  (!queriesData ||
    queriesData
      .slice(0, getQueryCount())
      .every(({ data }) => !data || (Array.isArray(data) && data.length === 0)));

[].every(...) is vacuously true, so the chart renders "No results were returned for this query" and the map never appears at all.

ChartMetadata already carries enableNoResults for exactly this situation, but only the native-filter path reads it; ChartRenderer derives the flag solely from server pagination:

const bypassNoResult = !(
  currentFormDataExtended?.server_pagination &&
  (hasSearchText || hasAgGridFilters)
);

Fix: honor getChartMetadataRegistry().get(vizType)?.enableNoResults there as well, and set enableNoResults: false on the Multiple Layers metadata. Also worth having transformProps not hand the container an undefined payload when queriesData is empty.

3. Layer lookup 404s for any principal whose chart access is scoped

fetchSubslices reads each layer with GET /api/v1/chart/<id>. Layer charts are saved charts that normally sit on no dashboard of their own, so ChartFilter denies them:

if (guest_dashboards := guest_embedded_dashboard_filter()) is not None:
    return query.filter(self.model.dashboards.any(guest_dashboards))
if security_manager.can_access_all_datasources():
    return query
return self._apply_viewers(query)

An embedded guest gets denied by the first branch (the layers are not on the token's dashboards) and a role without an explicit grant on the layer gets denied by _apply_viewers; in both cases even though the principal is entitled to the container. In our embedded dashboard every layer 404s, consistently.

Confirming it was the filter and not a stale id or a missing session, for one of our layers:

• GET /api/v1/chart/369 as an admin returns 200, so the chart exists and is readable.
• Its payload has "dashboards": []; it is a layer of a deck_multi chart, so it sits on no dashboard of its own. Slice.dashboards.any(<token dashboards>) can therefore never match it.
• The same request with no session returns {"msg": "Missing Authorization Header"}, not a 404. So the 404 we saw came from a request that was authenticated and scoped, which is the guest branch above.

Note this failure is deterministic rather than intermittent, so for embedded deployments it is the one that fully breaks the chart.

The legacy pipeline never hit this because it resolved the layers server-side, under the container's access:

slices = db.session.query(Slice).filter(Slice.id.in_(slice_ids)).all()

Fix: a GET /chart/<pk>/deck_layers/ endpoint that gates on the container (the normal base filter applies to that lookup) and then resolves the declared layers with skip_base_filter=True, exposing only their slice_id, viz_type and params. That reproduces the legacy rule, access to the container governs access to its layers, without widening the general chart filter. Multi.tsx uses it when the container is saved and falls back to the per-chart reads otherwise, so an unsaved chart in Explore still works.

Worth noting this also collapses N requests into one.

4. Sub-layer requests still take the async handoff

This is the one that cost me the most time, so it may be the most valuable. Multi.tsx posts with result_type: 'full', and in charts/data/api.py:

use_async = (
    is_feature_enabled("GLOBAL_ASYNC_QUERIES")
    and query_context.result_format == ChartDataResultFormat.JSON
    and query_context.result_type == ChartDataResultType.FULL
    and cache_timeout != CACHE_DISABLED_TIMEOUT
)
if use_async:
    return self._run_async(json_body, command, add_extra_log_payload)

So with GLOBAL_ASYNC_QUERIES enabled and a cold cache, each sub-layer request answers 202 with job metadata and no data. Multi.tsx posts directly with SupersetClient.post rather than through the chart pipeline, so it never registers a listener and cannot follow the job. The layer simply never renders, and the async middleware logs one of these per layer:

listener not found for job_id 9301681d-69e7-40c6-837a-83f1dcc9e7ce

It presents as intermittent and "fixed by reloading", because by the time you reload the job has completed and warmed the cache. That made it quite hard to chase.

Note this is the same failure the legacy path had on /explore_json/ -> moving to /api/v1/chart/data carries it along rather than resolving it.

Fix: request result_type: 'results' instead of 'full'. Both share the same preparer (None in _data_result_type_preparers), so it is the same query against the same cache; only the payload envelope is trimmed, and it still returns data / colnames / coltypes, which is all the layer transforms read (getRecordsFromQuery uses queriesData[0].data).

---

If it helps in prioritizing: #2 and #3 break the chart completely and deterministically; #2 for everyone, #3 for embedded deployments; while #4 presents as intermittent and self-healing, which made it by far the most expensive to diagnose.

[github-actions]

🎪 Showtime is building environment on GHA for 758cde9

[rusackas]

@sancho11 this is exactly the kind of testing this migration needs before it lands, thanks for tracing all this.

1 and 2 were already fixed on the branch, autozoom buckets by slice_id and enableNoResults: false skips the empty-state. Just pushed a fix for 4, switching to result_type: 'results' so it skips the async handoff you traced.

3 is real too, and the trickiest of the four, a new endpoint scoped to the container's own access rather than the per-chart check. Want to open that one against this branch like you offered? I'd rather have your production traces backing it than guess at it myself.

[rusackas]

@sancho11 went ahead and built 3 myself rather than leave it on your plate: GET /api/v1/chart/<pk>/deck_layers/, gated on the container, and Multi.tsx uses it when the chart's saved, falling back to per-chart reads in Explore. Thanks again for tracing all four of these.

[github-actions]

🎪 Showtime deployed environment on GHA for 1a2bee4

• Environment: http://34.222.17.87:8080 (admin/admin)
• Lifetime: 48h auto-cleanup
• Updates: New commits create fresh environments automatically

[sancho11]

@rusackas Thanks for picking 3 up; I looked at e04112f and it matches what I had locally, down to gating on the container and falling back to the per-chart reads in Explore, so there's nothing for me to add there. Glad the traces were useful.

There is a fifth gap I ran into afterwards, and I'd rather ask than assume where it belongs.

Composing layers loses the legend each of them shows standalone. DeckGLPolygon builds its metric buckets and CategoricalDeckGLContainer builds its category swatches, but both render the <Legend> from the layer's own React component, and Multi.tsx only ever calls getLayer, so those components are never mounted. The legend simply isn't there in Multiple Layers, and never has been. It's not something the migration broke.

I have it working locally against the migrated Multi.tsx: the categories are rebuilt per layer from the features its own transform returned, reusing the same getBuckets / getColorBreakpointsBuckets / getCategories helpers the components use rather than reimplementing the logic, so it follows if those change. Layers sharing a corner are merged into one legend instead of stacking on top of each other, and there's a show_legend control on the container so a multi-layer map can drop them without editing the layer charts it composes. Defaults to on, and only an explicit false hides them, so saved charts keep their legends.

This is a parity gap rather than a migration regression, though, and I don't want to grow the scope of a 616-file PR without checking. Happy either way. I can open it against this branch now, or hold it as a follow-up once this lands. Your call.

[rusackas]

The legend simply isn't there in Multiple Layers, and never has been. It's not something the migration broke.

I might look into it to see if it's an easy fix, but this PR is big and fragile enough that I'm inclined to tackle that on another PR. Feel free to file an Issue if you'd like.

@sancho11 It's a good catch, but if I go trying to solve all the pre-existing bugs with all the charts I've touched in this PR, I'll go crazy :)

[villebro]

LGTM. Despite the massive and welcome changes here (and quite frankly somewhat difficult to review), I feel great care has been taken to address potential issues. And not to forget all the multiple improvements. The benefits this brings IMO far outweigh any regression risks, hence I'm going to approve this. Great work @rusackas and HUGE kudos for seeing this through! Also big thanks to reviewers/testers!

[rusackas]

Thank you all for the help/support here.

[sancho11]

Kudos! 💪