fix(charts): exclude string helpers from post_processing operation allowlist by rusackas · Pull Request #43345 · apache/superset

rusackas · 2026-08-19T21:01:58Z

SUMMARY

Follow-up from a codeant-ai-for-open-source finding on #43337: the post_processing operation allowlist (ChartDataPostProcessingOperationSchema.operation in superset/charts/schemas.py, and the dispatch guard in QueryObject.exec_post_processing) accepts escape_separator/unescape_separator as valid operation names. Those are internal str -> str helpers used by flatten, not DataFrame post-processing operations — submitting one as an operation currently either raises a confusing TypeError (the function gets called with a DataFrame instead of a string) rather than a clean validation error.

This is pre-existing on master, independent of #43337 (the old inspect.getmembers(pandas_postprocessing, inspect.isfunction) allowlist has the same gap).

Adds pandas_postprocessing.OPERATIONS, a curated list of the real DataFrame operations (excludes the two string helpers), and uses it in both the schema allowlist and the executor's dispatch guard.

While adding a test for this, found and fixed a second, related bug in the same code path: the InvalidPostProcessingError message in query_object.py used type=operation against a "...%(operation)s" format string, so hitting that branch raised a KeyError from flask_babel's gettext instead of the intended clean error. Fixed to operation=operation.

TESTING INSTRUCTIONS

pytest tests/unit_tests/charts/test_schemas.py -k post_processing_operation
pytest tests/unit_tests/queries/query_object_test.py -k post_processing

Also ran the full tests/unit_tests/pandas_postprocessing/, tests/unit_tests/charts/, and tests/unit_tests/queries/ suites locally (349 passed, 2 xfailed, unrelated) and pre-commit on the changed files.

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

bito-code-review · 2026-08-19T21:02:09Z

Code Review Agent Run #8540c5

Actionable Suggestions - 0

Review Details

Files reviewed - 5 · Commit Range: dcbc142..dcbc142
- superset/charts/schemas.py
- superset/common/query_object.py
- superset/utils/pandas_postprocessing/__init__.py
- tests/unit_tests/charts/test_schemas.py
- tests/unit_tests/queries/query_object_test.py
Files skipped - 0
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers a full AI review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

codecov · 2026-08-19T21:11:01Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.19%. Comparing base (12cd259) to head (38a3b4c).
⚠️ Report is 42 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #43345   +/-   ##
=======================================
  Coverage   79.19%   79.19%           
=======================================
  Files        2879     2879           
  Lines      165870   165871    +1     
  Branches    38352    38352           
=======================================
+ Hits       131354   131355    +1     
  Misses      32039    32039           
  Partials     2477     2477

Flag	Coverage Δ
hive	`37.94% <66.66%> (+<0.01%)`	⬆️
mysql	`57.67% <66.66%> (+<0.01%)`	⬆️
postgres	`57.70% <66.66%> (+<0.01%)`	⬆️
presto	`39.85% <66.66%> (+<0.01%)`	⬆️
python	`83.74% <100.00%> (+<0.01%)`	⬆️
sqlite	`57.39% <66.66%> (+<0.01%)`	⬆️
unit	`73.96% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

bito-code-review · 2026-08-20T07:19:27Z

Code Review Agent Run #18cccd

Actionable Suggestions - 0

Review Details

Files reviewed - 5 · Commit Range: 30ae1b4..30ae1b4
- superset/charts/schemas.py
- superset/common/query_object.py
- superset/utils/pandas_postprocessing/__init__.py
- tests/unit_tests/charts/test_schemas.py
- tests/unit_tests/queries/query_object_test.py
Files skipped - 0
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers a full AI review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

…lowlist escape_separator/unescape_separator are str -> str helpers used internally by flatten, not DataFrame post-processing operations. Both the schema's validate.OneOf() choices and the query_object.py dispatch accepted them as a post_processing operation name, so submitting one against a DataFrame either raised a confusing TypeError (dispatch) or, previously, a KeyError in the fallback error path (the InvalidPostProcessingError message used type=operation against a %(operation)s format string). Adds pandas_postprocessing.OPERATIONS, a curated list of the real DataFrame operations, and uses it in both the schema allowlist and the executor's dispatch guard instead of the module's broader __all__/hasattr checks.

… from checked-in openapi.json Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

netlify · 2026-08-28T09:18:04Z

✅ Deploy Preview for superset-docs-preview ready!

Name	Link
🔨 Latest commit	`38a3b4c`
🔍 Latest deploy log	https://app.netlify.com/projects/superset-docs-preview/deploys/6a9151dc86ab9a00088385ee
😎 Deploy Preview	https://deploy-preview-43345--superset-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

bito-code-review · 2026-08-28T09:28:41Z

Code Review Agent Run #e11e55

Actionable Suggestions - 0

Review Details

Files reviewed - 5 · Commit Range: 8bde42f..38a3b4c
- superset/charts/schemas.py
- superset/common/query_object.py
- superset/utils/pandas_postprocessing/__init__.py
- tests/unit_tests/charts/test_schemas.py
- tests/unit_tests/queries/query_object_test.py
Files skipped - 1
- docs/static/resources/openapi.json - Reason: Filter setting
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers an incremental AI Review.
/review full - Manually triggers a full AI Review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

pull-request-size Bot added the size/M label Aug 19, 2026

dosubot Bot added the change:backend Requires changing the backend label Aug 19, 2026

rusackas mentioned this pull request Aug 19, 2026

feat(config): add EXTRA_PANDAS_POSTPROCESSING_OPS extension point #43337

Merged

9 tasks

rusackas requested review from betodealmeida, msyavuz and sadpandajoe and removed request for betodealmeida August 20, 2026 00:34

rusackas force-pushed the fix/post-processing-operation-allowlist branch from dcbc142 to 30ae1b4 Compare August 20, 2026 04:17

sadpandajoe reviewed Aug 21, 2026

View reviewed changes

Comment thread superset/charts/schemas.py Outdated

Comment thread superset/charts/schemas.py Outdated

rusackas force-pushed the fix/post-processing-operation-allowlist branch from 30ae1b4 to 8bde42f Compare August 28, 2026 06:25

fix(api): drop stale escape_separator/unescape_separator enum entries…

38a3b4c

… from checked-in openapi.json Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

github-actions Bot added the doc Namespace | Anything related to documentation label Aug 28, 2026

Navigation Menu

Uh oh!

fix(charts): exclude string helpers from post_processing operation allowlist - #43345

fix(charts): exclude string helpers from post_processing operation allowlist#43345
rusackas wants to merge 2 commits into
apache:masterfrom
rusackas:fix/post-processing-operation-allowlist

rusackas commented Aug 19, 2026

Uh oh!

bito-code-review Bot commented Aug 19, 2026 •

edited

Loading

Code Review Agent Run #8540c5

Uh oh!

codecov Bot commented Aug 19, 2026 •

edited

Loading

Uh oh!

bito-code-review Bot commented Aug 20, 2026 •

edited

Loading

Code Review Agent Run #18cccd

Uh oh!

Uh oh!

Uh oh!

netlify Bot commented Aug 28, 2026

Uh oh!

bito-code-review Bot commented Aug 28, 2026 •

edited

Loading

Code Review Agent Run #e11e55

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

rusackas commented Aug 19, 2026

SUMMARY

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Uh oh!

bito-code-review Bot commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Agent Run #8540c5

Uh oh!

codecov Bot commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bito-code-review Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Agent Run #18cccd

Uh oh!

Uh oh!

Uh oh!

netlify Bot commented Aug 28, 2026

✅ Deploy Preview for superset-docs-preview ready!

Uh oh!

bito-code-review Bot commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Agent Run #e11e55

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bito-code-review Bot commented Aug 19, 2026 •

edited

Loading

codecov Bot commented Aug 19, 2026 •

edited

Loading

bito-code-review Bot commented Aug 20, 2026 •

edited

Loading

bito-code-review Bot commented Aug 28, 2026 •

edited

Loading