fix(forecast): resolve time grain robustly for Prophet forecasting - #42145
Conversation
|
Bito Review Skipped - Source Branch Not Found |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #42145 +/- ##
==========================================
- Coverage 65.17% 64.97% -0.20%
==========================================
Files 2753 2784 +31
Lines 154657 157312 +2655
Branches 35484 35841 +357
==========================================
+ Hits 100801 102221 +1420
- Misses 51940 53122 +1182
- Partials 1916 1969 +53
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR fixes a forecasting failure mode where the Prophet post-processing step could be invoked without a time_grain, causing an unhandled backend TypeError (500) instead of a readable validation error. It makes time grain resolution more robust on the frontend and makes the backend Prophet post-processing function resilient to missing time_grain in the dispatched options.
Changes:
- Frontend: resolve the effective
time_grainfrom (1) adhoc x-axis columntimeGrain, (2)queryObject.extras.time_grain_sqla, (3)formData.time_grain_sqla, falling back to daily (P1D). - Backend: make
time_grainoptional inprophet()so missing/Nonehits the existing"Time grain missing"InvalidPostProcessingErrorpath rather than raising a rawTypeError. - Tests: add regression coverage for missing/
Nonetime_grainon the backend, and for the new frontend resolution/fallback behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
tests/unit_tests/pandas_postprocessing/test_prophet.py |
Adds regression tests ensuring missing/None time_grain yields a readable InvalidPostProcessingError. |
superset/utils/pandas_postprocessing/prophet.py |
Updates Prophet post-processing signature to accept optional time_grain and preserve graceful error behavior. |
superset-frontend/packages/superset-ui-chart-controls/src/operators/prophetOperator.ts |
Resolves effective time_grain from x-axis/query extras/form data with a P1D fallback to avoid dropped undefined options. |
superset-frontend/packages/superset-ui-chart-controls/test/operators/prophetOperator.test.ts |
Adds tests for time grain resolution sources and the daily fallback when no grain is provided. |
|
🎪 Showtime deployed environment on GHA for 6c376b7 • Environment: http://34.214.238.14:8080 (admin/admin) |
rusackas
left a comment
There was a problem hiding this comment.
Thanks @yousoph, LGTM.
Curious about the P1D fallback: for a chart where Time Grain was deliberately cleared to None, assuming daily could put the forecasted points at the wrong cadence versus the actual data. Not blocking, just wondering if that's worth a follow-up.
6c376b7 to
0338d53
Compare
✅ Deploy Preview for superset-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
🎪 Showtime deployed environment on GHA for 0338d53 • Environment: http://54.149.38.146:8080 (admin/admin) |
…pache#42145) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(mcp): dedupe list-tool schemas and delete dead middleware (apache#41923) * fix(ag-grid-table): respect row limit with server pagination (apache#41346) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(table/pivot-table): correct non-additive totals/subtotals via DB rollup [SIP-216] (apache#41184) Co-authored-by: Superset Dev <dev@superset.apache.org> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Amin Ghadersohi <amin.ghadersohi@gmail.com> * feat(datasets): add RLS filter indicator badge to dataset list and explore view (apache#38807) Co-authored-by: Evan <evan@preset.io> * fix(chart): updates counties of kenya map (apache#38019) Co-authored-by: Zack Adams <zack@Zacks-Laptop.local> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Evan <evan@preset.io> Co-authored-by: Evan Rusackas <evan@rusackas.com> * chore(deps): bump actions/setup-go from 6.5.0 to 7.0.0 (apache#42303) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump nanoid from 5.0.9 to 6.0.0 in /superset-frontend (apache#42230) Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: hainenber <dotronghai96@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: hainenber <dotronghai96@gmail.com> Co-authored-by: Joe Li <joe@preset.io> Co-authored-by: Evan <evan@preset.io> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> * test(dashboard): migrate dashboard load smoke test to Playwright (apache#41432) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(superset_app_root): when used with oauth (apache#38033) Signed-off-by: Grégoire Bellon-Gervais <gregoire.bellon-gervais@docaposte.fr> Co-authored-by: Evan Rusackas <evan@preset.io> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * refactor: engine enforce SQLAlchemy 2.0 (apache#42277) * fix(ag-grid-table): avoid ambiguous build query import (apache#42313) * fix: Revert "chore(deps): bump echarts from 5.6.0 to 6.1.0 in /superset-frontend" (apache#42314) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * adding circleci config * fix(plugin-chart-echarts): import the -obj locale build so time axes render (apache#42317) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(pandas_postprocessing): avoid FutureWarning for max/min in boxplot MINMAX (apache#42272) * fix(heatmap): correct tooltip axis value lookup and percentage calculations and add tests (apache#41864) Signed-off-by: yousoph <sophieyou12@gmail.com> Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(forecast): resolve time grain robustly for Prophet forecasting (apache#42145) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix: Added PostgreSQL 17.X to the supported database versions table in (apache#42280) * fix(helm): add MCP HTTPRoute configuration (apache#42219) * fix(dashboard): offer Exit edit mode when there is nothing to discard (apache#42208) Co-authored-by: Claude Code <noreply@anthropic.com> * chore(deps): bump actions/labeler from 6.2.0 to 7.0.0 (apache#42332) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump github/codeql-action/analyze from 4.37.0 to 4.37.1 (apache#42331) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump github/codeql-action/init from 4.37.0 to 4.37.1 (apache#42334) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps-dev): bump @formatjs/intl-durationformat from 0.10.17 to 0.10.18 in /superset-frontend (apache#42337) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump caniuse-lite from 1.0.30001805 to 1.0.30001806 in /docs (apache#42333) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump echarts to 6.1.0 with locale and containLabel guards (apache#42315) (apache#42321) Co-authored-by: Claude Code <noreply@anthropic.com> Co-authored-by: Amin Ghadersohi <amin.ghadersohi@gmail.com> * docs: add pattern to the list of organisations using superset (apache#42341) * chore(deps): bump ag-grid from 36.0.0 to 36.0.1 in /superset-frontend (apache#42338) Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: hainenber <dotronghai96@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: hainenber <dotronghai96@gmail.com> * fix(mcp): trust dataset is_dttm flag when applying time_grain to VARCHAR temporal columns (apache#42288) * chore(deps-dev): update taos-ws-py requirement from >=0.6.9 to >=0.7.0 (apache#42344) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps-dev): bump the storybook group in /superset-frontend with 5 updates (apache#42355) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(native-filters): keep filter value input caret at inline start (apache#42323) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(mcp): truncate query-tool responses instead of hard-failing (apache#42244) * chore(deps): bump nh3 from 0.3.5 to 0.3.6 (apache#42349) Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Joe Li <joe@preset.io> * chore(deps): bump pydantic from 2.11.7 to 2.13.4 (apache#42350) Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Joe Li <joe@preset.io> * chore(deps): bump sqlalchemy-continuum from 1.6.0 to 1.7.0 (apache#42351) Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Joe Li <joe@preset.io> * fix(embedded): stop rejecting guest chart data built from control-specific params keys (apache#42295) Co-authored-by: Claude Code <noreply@anthropic.com> * fix(explore): show the beginning date on time-series x-axis line charts (apache#42046) * fix(api): add example to get_export_ids_schema so Swagger "Try it out" pre-fills a valid array (apache#42265) * chore: SQLAlchemy User cascade backref warnings are irrelevant (apache#42360) * fix(charts): handle async (202) chart-data responses in StatefulChart (apache#42157) Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com> Co-authored-by: Evan Rusackas <evan@preset.io> * feat(KustoKQL): Add support for NULL / IS NOT NULL operator (apache#37890) Co-authored-by: ag-ramachandran <ramacg@microsoft.com> Co-authored-by: Joe Li <joe@preset.io> * chore(deps): bump pillow from 12.2.0 to 12.3.0 (apache#42348) Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Joe Li <joe@preset.io> * chore(deps): bump flask-compress from 1.17 to 1.24 (apache#42346) Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Joe Li <joe@preset.io> * chore(deps-dev): bump databricks-sql-connector from 4.2.6 to 4.3.0 (apache#42347) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Joe Li <joe@preset.io> Co-authored-by: Evan Rusackas <evan@preset.io> * fix(native-filters): use FILTER_STATE_CACHE_CONFIG timeout for dynamic filter option queries (apache#38910) * fix(explore): render Jinja before validating legacy chart filters (apache#41996) * fix(dataset): disable duplicate button when name is empty (apache#42217) Co-authored-by: AS-MAC-1123 <as-mac-1123@AS-MAC-1123.local> Co-authored-by: Evan Rusackas <evan@rusackas.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * docs(gunicorn): correct dead links in values.yaml (apache#42385) * chore(deps): bump @deck.gl/mapbox from 9.3.6 to 9.3.7 in /superset-frontend in the deckgl group (apache#42377) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/checkout from 7.0.0 to 7.0.1 (apache#42376) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump immer from 11.1.11 to 11.1.15 in /superset-frontend (apache#42378) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump body-parser from 1.20.5 to 1.20.6 in /docs (apache#42370) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps-dev): bump brace-expansion from 1.1.15 to 1.1.16 in /superset-embedded-sdk (apache#42369) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(security): bump pyasn1 from 0.6.3 to 0.6.4 (apache#42363) * fix(security): bump pillow from 12.2.0 to 12.3.0 (apache#42362) * docs(map-tiles): add Yandex Maps Tiles API configuration (apache#42375) * ci: improve conditional checks for lillio tests and build --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: hainenber <dotronghai96@gmail.com> Signed-off-by: Grégoire Bellon-Gervais <gregoire.bellon-gervais@docaposte.fr> Signed-off-by: yousoph <sophieyou12@gmail.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Amin Ghadersohi <amin.ghadersohi@gmail.com> Co-authored-by: Evan Rusackas <evan@preset.io> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Superset Dev <dev@superset.apache.org> Co-authored-by: SkinnyPigeon <e.blackledge@stuart.com> Co-authored-by: Zack <adams.z.d@gmail.com> Co-authored-by: Zack Adams <zack@Zacks-Laptop.local> Co-authored-by: Evan Rusackas <evan@rusackas.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: hainenber <dotronghai96@gmail.com> Co-authored-by: Joe Li <joe@preset.io> Co-authored-by: Grégoire <greggbg@gmail.com> Co-authored-by: Hans Yu <hans.yu@outlook.de> Co-authored-by: Elizabeth Thompson <eschutho@gmail.com> Co-authored-by: yousoph <sophieyou12@gmail.com> Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com> Co-authored-by: Amitesh Gupta <143833521+singlaamitesh@users.noreply.github.com> Co-authored-by: David <39565245+dmunozv04@users.noreply.github.com> Co-authored-by: Yash Shrivastava <119301033+alephys26@users.noreply.github.com> Co-authored-by: JUST.in DO IT <justin.park@airbnb.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: jesperct <jmilecelento@gmail.com> Co-authored-by: Abdul Rehman <76230556+Abdulrehman-PIAIC80387@users.noreply.github.com> Co-authored-by: jenwitteng <jenwit.amonpongitsara@agoda.com> Co-authored-by: Ramachandran A G <106139410+ag-ramachandran@users.noreply.github.com> Co-authored-by: ag-ramachandran <ramacg@microsoft.com> Co-authored-by: Ujjwal Jain <jainujjwal1609@gmail.com> Co-authored-by: Jean Massucatto <massucattoj@gmail.com> Co-authored-by: suvankardas216 <rohanrohan510@gmail.com> Co-authored-by: AS-MAC-1123 <as-mac-1123@AS-MAC-1123.local> Co-authored-by: Alejandro Solares <219859296+ASolarers-Rodriguez@users.noreply.github.com> Co-authored-by: ViktorGo86 <114023094+ViktorGo86@users.noreply.github.com>
SUMMARY
Enabling Forecast (Advanced Analytics → Predictive Analytics) on a time-series chart could fail with an opaque 500:
Root cause: the
prophetpost-processing op is dispatched viagetattr(pandas_postprocessing, operation)(df, **options)inQueryObject.exec_post_processing. The frontendprophetOperatorsettime_grain: formData.time_grain_sqla; when that value isundefined— e.g. a saved/dashboard chart whose Time Grain control was cleared to "None", or a chart using the generic x-axis where the grain lives on the adhoc column'stimeGrainpopover instead of the panel control —JSON.stringifydrops the key. The backend then callsprophet()withouttime_grainand Python raises a rawTypeErrorat call time, before the gracefulif not time_grain: raise InvalidPostProcessingError(_("Time grain missing"))guard insideprophet()can run.Fix (both ends):
prophetOperator.ts): resolve the effective time grain in priority order — adhoc x-axis columntimeGrain→queryObject.extras.time_grain_sqla(picks up dashboard-applied grains) →formData.time_grain_sqla→P1Dfallback (matching thetime_grain_sqlacontrol default insharedControls), so forecasting works instead of erroring.prophet.py): maketime_grainan optional kwarg (Optional[str] = None), so any remaining path that can't determine a grain surfaces the existing readable "Time grain missing"InvalidPostProcessingError(400) instead of an unhandledTypeError(500). No positional callers exist — the op is always dispatched with keyword options.BEFORE/AFTER
Before: forecast on a chart without
time_grain_sqlain its form data → HTTP 500, rawTypeErrormessage shown in the chart.After: the grain is resolved from the x-axis column / query extras / form data (falling back to
P1D) and the forecast renders; if a grain still can't be determined server-side, the user gets a readable "Time grain missing" error (400).TESTING INSTRUCTIONS
New tests:
time_grainand explicittime_grain=Noneboth raiseInvalidPostProcessingError("Time grain missing") instead ofTypeError.queryObject.extras, and theP1Dfallback when no grain is present anywhere.To reproduce manually: save a Time-series Line Chart, clear Time Grain to "None", enable Forecast under Advanced Analytics, run the chart.
ADDITIONAL INFORMATION
🤖 Generated with Claude Code