Skip to content

test: seed MSQ quantiles sketch for partition tests - #20440

Merged
FrankChen021 merged 1 commit into
apache:masterfrom
gitedmond:test/seed-msq-quantiles-sketch
Sep 28, 2026
Merged

FrankChen021 merged 1 commit into
apache:masterfrom
gitedmond:test/seed-msq-quantiles-sketch

Conversation

@gitedmond

Copy link
Copy Markdown
Contributor

Fixes #20439.

Description

ClusterByStatisticsCollectorImplTest fixes the input shuffle order, but the DataSketches ItemsSketch also uses a shared random generator when it compacts and merges. Different sketch samples can push a partition outside the test's fixed weight leeway.

Seed the sketch generator before each test method so the existing partition coverage, boundary, and weight assertions run against a repeatable sample. This changes only the test; production partitioning is unchanged. Seed 0L reproduced the same weight failure for this fixture, so the test uses 1L after checking the target method and the full class.

Verification

  • Before the change, with Surefire retries disabled, the target method failed on the second unseeded run: single collector, random order, generatePartitionsWithMaxCount(50), partition #13, at verifyPartitionWeights.
  • After the change, all 15 tests in ClusterByStatisticsCollectorImplTest passed with retries disabled.
  • The formerly flaky method passed five additional isolated runs with retries disabled.
  • git diff --check passed.

Tests used JDK 25 and Maven 3.9.16 with -pl multi-stage-query -am, -Pskip-static-checks, and -Dweb.console.skip=true.


Key changed/added classes in this PR
  • ClusterByStatisticsCollectorImplTest

This PR has:

  • been self-reviewed.

@github-actions github-actions Bot added Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Sep 28, 2026

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No actionable issues found. The change is limited to the test class and uses DataSketches' documented deterministic-test random hook; resetting it before each test makes each method repeatable without changing production behavior. I inspected the full changed class, the collector and sketch call paths, the pinned DataSketches 4.2.0 source/API, module test execution settings, related MSQ tests, and the existing ItemsSketch test pattern.

Reviewed 1 of 1 changed files.

Validation: Static review completed; git diff --check passed. No build or test commands were run.


This is an automated review by Codex GPT-5.6-Luna(max)

@FrankChen021
FrankChen021 merged commit 1afd36b into apache:master Sep 28, 2026
27 checks passed
@github-actions github-actions Bot added this to the 39.0.0 milestone Sep 28, 2026
@gitedmond
gitedmond deleted the test/seed-msq-quantiles-sketch branch September 28, 2026 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 Flaky test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky test: ClusterByStatisticsCollectorImplTest.test_clusterByXYbucketByX_threeX_uniqueY

2 participants