Repository navigation
Flaky test Triage - 2026-09-24 #20414
Description
This issue was generated automatically by Claude Code (Anthropic's AI coding agent) running a scheduled CI-triage routine on behalf of @FrankChen021. Analysis and suggested fixes are AI-produced; please verify before acting on them.
This triage covers the 8 commits merged to master on 2026-09-23 (local time, UTC+8). Three commits had at least one failed push-triggered job (four failed jobs in total). None of the failures is caused by the commit itself: all are flaky tests or a CI infrastructure hiccup. Each failing shard passed on the PR's own pre-merge run and on the neighbouring master commits, and none of the three commits touches the failing code (two add Google/Azure deep-storage features, one bumps nimbus-jose-jwt). No re-runs had been triggered on any failed job.
Summary
| Commit | Failed job | Failure log | Root cause | Verdict |
|---|---|---|---|---|
| a379ea8 (#20397, build(deps): bump com.nimbusds:nimbus-jose-jwt from 10.9.1 to 10.10) | openrewrite |
job 107010233252 | Maven could not resolve the parent POM org.apache:apache:pom:39 ("Could not transfer artifact ... from/to ${repoOrgId} (${repoOrgUrl})"), so the build aborted before OpenRewrite ran |
Infra |
| a379ea8 (#20397, build(deps): bump com.nimbusds:nimbus-jose-jwt from 10.9.1 to 10.10) | unit tests (25, K*,U*,Z*,Y*,X*) |
job 107010233701 |
KinesisFaultToleranceTest.test_supervisorRecovers_afterSuspendResume_withEmptyShards: SELECT COUNT(*) returned 2001 vs expected 3000 right after 3000 rows were reported published, failed 1/1 attempt |
Flaky (race) |
| 15051a5 (#20382, push/pull/kill segments in GCS without zip) | unit tests (25, H*,D*,T*,O*) |
job 106879002642 |
TaskQueueScaleTest.doMassLaunchAndExit: running+pending+waiting count 995/997/998/998 vs expected 1000, failed all 4 surefire attempts |
Flaky (race) — Resolved (#20291) |
| 4eede66 (#20380, Azure SegmentRangeReader for partial loads) | unit tests (25, K*,U*,Z*,Y*,X*) |
job 106878085304 |
KinesisFaultToleranceTest.test_supervisorRecovers_afterChangeInTopicPartitions_withEmptyShards: timed out after 240s waiting for ingest/rows/published to reach 2000 in verifyAndTearDown, failed 1/1 attempt |
Flaky |
Analysis and suggested fixes
Status of each item updated on 2026-09-26.
Open
1. KinesisFaultToleranceTest.test_supervisorRecovers_afterSuspendResume_withEmptyShards (embedded-tests, K* shard)
StreamIndexTestBase.verifyRowCount still gates on waitForAllSegmentsToBeAvailable, which counts schema-refresh metrics. That count can be reached before the handed-off segments are queryable, so COUNT(*) returned 2001 instead of 3000.
Suggested fix: switch verifyRowCount to EmbeddedClusterApis.waitForAllSegmentsToBeQueryable (added in #20319), or poll COUNT(*) until it matches.
2. KinesisFaultToleranceTest.test_supervisorRecovers_afterChangeInTopicPartitions_withEmptyShards (embedded-tests, K* shard)
After resharding from 2 to 4 shards, ingest/rows/published did not reach 2000 within 240 s.
Suggested fix: wait for the supervisor to report 4 partitions before waiting on the row aggregate, and log the supervisor status and the current row sum on timeout.
3. Maven parent POM org.apache:apache not resolvable (${repoOrgUrl})
A transient Central or cache miss makes Maven fall back to a repository with unresolved placeholders, so the build aborts before any test runs. It happened in openrewrite on 2026-09-23 (#20414) and in QTest 2/4 on 2026-09-25 (#20428).
Suggested fix: pre-resolve org.apache:apache in the ~/.m2 warm-up step with retries, delete *.lastUpdated files before the build, and enable HTTP retries in .mvn/maven.config.
Resolved
4. TaskQueueScaleTest.doMassLaunchAndExit: Resolved by #20291
The test summed separate running, pending and waiting snapshots. #20291 (merged 2026-09-26) removes that assertion. No action needed.
Activity
FrankChen021 commentedon Sep 26, 2026
Superseded by per-problem tracking issues and the pinned dashboard #20437. Remaining open problems from this report: #20433, #20434, #20436. Resolved items are listed in the dashboard.