Bug triage results: 2026-09-21

[andygrove]

Triage pass over the open requires-triage queue, per the project Bug Triage Guide.

• Date: 2026-09-21
• Total issues processed: 57 (50 triaged, 7 skipped, 0 failed)
• Type counts: 15 bugs, 35 enhancements
• Priority counts applied: priority:critical 5, priority:high 1, priority:medium 6, priority:low 3
• Guide: docs/source/contributor-guide/bug_triage.md

Labels have already been applied. A reviewer should spot-check the calls below and close this issue when satisfied; corrections should be made directly on the affected issue.

Notes on this pass:

• Three type reclassifications are worth a second look, all from enhancement to bug: #6024, #6022 and #6060. The first two were filed by their author with the feature-request template and an enhancement label already set. They were reclassified because in both the query fails under Comet where plain Spark succeeds, which the guide places under priority:medium "broken features that have workarounds" rather than under new functionality. If the intent was to track them as new resilience features, flipping them back is a one-line edit.
• One priority escalation: #6002 was raised from the author's priority:high to priority:critical; see Escalations to consider.
• The guide's area table is out of date. The repository now carries area:memory, area:joins and area:Iceberg, none of which appear in the guide, so this pass did not apply them — the same note was made in the 2026-08-17, 2026-08-31 and 2026-09-14 passes. That gap cost real labelling this time: #5997, #5961, #6028 and #6032 have no area label but are plainly area:memory, and #6056, #6013, #6008 and #5962 are plainly area:joins. Adding the three labels to the guide's area table would fix this for the next pass.
• The regression label now exists in the repository but is not in the guide, so this pass did not apply it. That is the label proposed by #5925; the remaining work in that issue is the guide section and the skill step, which is why it is still open and classified as an enhancement here.
• The guide lists spark 4 as a pre-existing area indicator, but the repository only has spark 4.0 / spark 4.1 / spark 4.2 / spark 3.x, so no spark 4 label was applied.
• Where an issue already carried non-guide labels (correctness, performance, documentation, area:memory, area:shuffle), those were left untouched and are included in the area lists below for context.
• No enhancement in this queue arrived with an author-applied priority label, so nothing needed stripping.

Bugs

priority:critical

• Variance and standard deviation return incorrect results for large nearby values (#6044)
  • Area labels: area:aggregation
  • Rationale: var_pop returns 2.0 where Spark returns 1.0, a 100% relative error with no error or fallback, which is step 1 of the decision tree.
• Sliding-window SUM(BIGINT) ignores ANSI and TRY overflow semantics (#6043)
  • Area labels: area:aggregation
  • Rationale: try_sum returns a wrapped negative value where Spark returns NULL and ANSI sum returns that value instead of raising, so the wrong answer is delivered silently in both modes.
• Nested floating-point IN membership does not match Spark for signed zero (#6019)
  • Area labels: area:expressions
  • Rationale: the native nested membership comparator returns false where Spark's interpreted ordering returns true for -0.0 versus 0.0, with no error, which is the guide's silent-wrong-results case.
• SUM(decimal) returns NULL or throws under ANSI when an intermediate sum overflows the buffer precision but the final result fits (#6002)
  • Area labels: area:aggregation, correctness
  • Rationale: in legacy mode Comet returns NULL where Spark returns 0.6, a silent wrong result at step 1 of the decision tree; escalated from the author's priority:high (see escalations below).
• Iceberg serde: residual reflection failures can silently drop residual filtering (#5992)
  • Area labels: area:scan
  • Rationale: an unexpected reflection failure is indistinguishable from an intentional non-push, so the task is serialized without the residual predicate and can return extra rows with no error — step 1 asks whether the bug can cause silent wrong results, and here it can whenever no exact filter sits above the scan.

priority:high

• TRY_CAST on narrowing map keys fails where Spark returns a map with a null key (#5995)
  • Area labels: area:expressions
  • Rationale: on main the non-nullable key field makes StructArray::new panic inside arrow and abort the executor, which is the guide's step 2 crash case on a plan Comet itself admitted as Compatible.

priority:medium

• Native S3 scan on EKS/IRSA turns a transient STS throttle into a hard 403 storm (#6024)
  • Area labels: area:scan
  • Rationale: the credential chain does not retry the throttled assume-role call and silently downgrades to the node instance role, so the job dies with 403s where plain Spark's S3A retry survives; loud rather than silent, and workable around by lowering per-pod concurrency, so step 3 rather than step 1 or 2. Reclassified from enhancement.
• Native Parquet S3 scan fails on credential provider classes that Spark/Hadoop accept (#6022)
  • Area labels: area:scan
  • Rationale: the query fails with Unsupported credential provider where plain Spark runs it fine, and Comet's own Iceberg path succeeds on the same cluster — a broken feature with a workaround, which the guide's priority:medium row covers under "missing expression support, writer feature gaps". Reclassified from enhancement.
• Legacy mode negation of a scalar signed minimum errors where the array path wraps (#6015)
  • Area labels: area:expressions
  • Rationale: the scalar arm raises an arithmetic overflow error in legacy mode where Spark and Comet's own array arm both wrap, failing the query rather than returning a wrong value, so step 3.
• Columnar shuffle rejects range partitioning based on a native-serde check it never uses (#5971)
  • Area labels: area:shuffle
  • Rationale: the reporter states it is an unnecessary-fallback bug rather than a correctness bug — the serialized SortOrder is never sent to native code on the columnar path, so Comet's shuffle is disabled for orderings the JVM would have partitioned correctly.
• fair_unified memory pool caps the pool total at pool_size / num_consumers instead of each consumer's share (#5961)
  • Area labels: area:memory
  • Rationale: the default off-heap pool grants a whole task only pool_size / num_consumers and tightens as consumers register, so work spills or fails early; a functional regression in the default configuration with a workaround (select another pool). Introduced by the DataFusion 53 upgrade (deps: upgrade to DataFusion 53.0, Arrow to 58.1 #3629), which predates the 1.0.0 tag, so it is not a regression against the most recent release.
• Field id gating differs from Spark: root-only check and no dependence on fieldId.read.enabled (#5936)
  • Area labels: area:scan
  • Rationale: two divergences, neither of which is an established wrong value — Comet returns name-matched rows where Spark raises, and Comet rejects a file whose ids are only on nested fields where Spark null-fills. Kept at step 3; escalate if someone shows the rows returned in the first case differ from the stored data.

priority:low

• Parquet writer tests can capture a previous write's plan (#6075)
  • Area labels: area:writer
  • Rationale: captureWritePlan accepts a delayed callback from a setup write, producing a false assertion failure; test-only, which is step 4.
• ci: a transient Maven Central failure in lint-java is reported as a scalafix violation (#6060)
  • Area labels: area:ci
  • Rationale: a plugin-resolution hiccup fails the job and turns Required Checks red with no source file at fault; CI tooling, so step 4, but see escalations.
• Nightly Miri workflow has failed every night since 2026-07-01 (#5999)
  • Area labels: area:ci
  • Rationale: 79 consecutive scheduled-run failures, none of them in Comet's own unsafe code and none blocking a pull request; the cost is a lost signal rather than a broken build, which is step 4.

Enhancements

• Add native Vortex scan support (#6070)
  • Area labels: area:scan
  • Rationale: proposes a new optional contrib module and native reader for a format Comet does not read today.
• Remove memory accounting from Comet's on-heap mode (#6063)
  • Area labels: area:shuffle, area:memory
  • Rationale: removal of a second accounting implementation that protects nothing — a refactor, with no reported wrong behaviour in the supported off-heap configuration.
• Native Iceberg scan: support abfs/abfss (Azure Data Lake Storage Gen2) locations (#6058)
  • Area labels: area:scan
  • Rationale: the scheme is declined at planning time and the scan runs on the JVM exactly as without the plugin, so this is new coverage rather than a defect; the backend is already compiled in.
• Allow row-based build sides (such as JDBC) to convert to Arrow for native BroadcastHashJoin (#6056)
  • Area labels: none (area:joins would apply if the guide adopted it)
  • Rationale: asks for a new row-to-Arrow bridge at the broadcast boundary; today's behaviour is a correct fallback, not a defect.
• Audit Comet logical-link repair across AQE replanning (#6034)
  • Area labels: none
  • Rationale: a design audit that explicitly "does not assume all repair is removable or establish another failing exchange case".
• Tuning guide does not explain that Comet's memory comes out of the executor container (#6032)
  • Area labels: documentation (area:memory would apply if the guide adopted it)
  • Rationale: documentation addition, which the guide's type table places under enhancement.
• Memory management cgroup diagram does not show which config value sizes each region (#6028)
  • Area labels: none (area:memory would apply if the guide adopted it)
  • Rationale: documentation improvement to an existing diagram.
• perf: broadcast hash join rebuilds the build-side hash table in every task (#6013)
  • Area labels: performance (area:joins would apply if the guide adopted it)
  • Rationale: a performance optimization request — the results are correct, the work just scales with task count instead of executor count.
• Evaluate rotating the native shuffle spill file at a byte threshold (#6010)
  • Area labels: area:shuffle
  • Rationale: asks for a measured comparison of three spill-file layouts; no failure is reported against the current one.
• Free and account for spill range metadata in the native shuffle writer (#6009)
  • Area labels: area:shuffle
  • Rationale: a memory-accounting and hygiene improvement with a computed worst case and no observed failure, consistent with how the other accounting-gap issues in this queue (Improve memory accounting: bound the JVM Arrow allocator and make native reclaim work #5997, Remove memory accounting from Comet's on-heap mode #6063) are classified.
• Text-format source on a broadcast build side blocks native BroadcastHashJoin (#6008)
  • Area labels: none (area:joins would apply if the guide adopted it)
  • Rationale: asks either for auto-inserted CometSparkToColumnar on broadcast build sides or a native Text reader; both are new capability, and the issue notes the interim config workaround already works.
• Track evaluation masks for data-dependent errors beyond unbase64 operator fallback (#6006)
  • Area labels: area:expressions
  • Rationale: explicitly "does not claim that every expression/context combination below is currently broken" — it tracks the audit that would establish which, if any, are.
• Global aggregate with no grouping keys and no aggregate functions falls back to Spark (#6001)
  • Area labels: area:aggregation
  • Rationale: a correct fallback with a [COMET: ...] reason, costing coverage and two transitions rather than correctness.
• Improve memory accounting: bound the JVM Arrow allocator and make native reclaim work (#5997)
  • Area labels: none (area:memory would apply if the guide adopted it)
  • Rationale: collects four accounting improvements ordered by cost, after feat: add optional container memory guard based on cgroup usage #5993 established that thresholding a cgroup counter does not work.
• Support Spark-compatible native hash and xxhash64 for decimals with precision >18 (#5994)
  • Area labels: area:expressions, performance
  • Rationale: the SQL admission guard already rejects these types recursively, so results are protected today and this is new native coverage.
• Nightly SNAPSHOT publishing is blocked until ASF Infra provisions Nexus credentials (#5985)
  • Area labels: area:ci
  • Rationale: tracks the completion of the publishing feature added in ci: publish nightly SNAPSHOT jars to repository.apache.org #5902 once INFRA-28409 is resolved; the 401 is the absence of credentials, not a defect in the workflow.
• [Variant] Preserve typed-value precedence without residual pre-rewriting (#5980)
  • Area labels: area:scan
  • Rationale: moves an existing, working decision earlier in the pipeline so the temporary null bitmap and replacement residual column from feat: support direct Variant projection in native Parquet scans #5868 can be removed.
• [Variant] Replace missing-key metadata repair with native permissive reconstruction (#5979)
  • Area labels: area:scan
  • Rationale: replaces the extend_shredded_metadata workaround with a writable output dictionary; the Arrow panic it cites is guarded by the current repair.
• [Variant] Remove the extra Spark byte-reconstruction pass (#5978)
  • Area labels: area:scan
  • Rationale: folds a second pass into the original traversal — a performance and structure change over behaviour that is already correct.
• [Variant] Consolidate Spark-compatible missing-value validation and errors (#5977)
  • Area labels: area:scan
  • Rationale: moves existing checks into the recursive unshredder behind an opt-in policy, keeping Arrow's specification-conforming default.
• Audit remaining native Partial-to-Spark Final aggregate compatibility (#5975)
  • Area labels: area:aggregation
  • Rationale: recovers native coverage that fix: revert unsafe partial aggregates after final fallback #5421 deliberately withdrew; the withdrawal is the safe state, so this is coverage work rather than a defect.
• ci: cancel merge queue runs on the first job failure (#5969)
  • Area labels: area:ci
  • Rationale: a CI efficiency improvement — the queue already reaches the right verdict, it just wastes roughly an hour of runner time doing so.
• test: cover Java float rendering through SQL file tests and consolidate the Iceberg partition path suites (#5968)
  • Area labels: area:expressions
  • Rationale: two deferred test-placement suggestions from the fix: render float and double Iceberg partition values like iceberg-java #5840 review that restructure tests rather than change behaviour.
• Forward Spark-generated runtime Bloom predicates into native Parquet readers (#5965)
  • Area labels: area:scan
  • Rationale: the issue states it is an optimization gap and that the existing residual filter and join preserve query results.
• Investigate mixed-type duplicate Parquet roots: Spark field-ID reads return anomalous values (#5964)
  • Area labels: area:scan
  • Rationale: the anomalous [1,0,null] is Spark's own output, and the contrasting Comet result came from an unpublished experimental candidate, so no defect in released Comet is established; this tracks the investigation that would define the target behaviour.
• Compute safe reusable expressions before broadcast materialization (#5962)
  • Area labels: area:expressions (area:joins would also apply if the guide adopted it)
  • Rationale: a planning optimization with an explicit no-change path for unsupported cases.
• Delegate Delta physical schema preparation to DeltaParquetFileFormat.prepareSchemaForRead (#5949)
  • Area labels: area:scan
  • Rationale: removes duplication of logic Delta already implements; a follow-up from the feat: add native Delta Lake scan contrib module (page/row-group pruning) #5365 review with the existing fallbacks staying in place.
• Route Delta deletion-vector metadata preparation through the scan's instrumented reader (#5948)
  • Area labels: area:scan
  • Rationale: the shared metadata cache is filled correctly; what is missing is that the preparation I/O lands in the scan's counters, which is an instrumentation improvement.
• Add envelope and provider contract tests for the Delta plan-data injector (#5947)
  • Area labels: area:scan
  • Rationale: adds tests to pin an existing boundary; marked non-blocking in the feat: add native Delta Lake scan contrib module (page/row-group pruning) #5365 review.
• Make the Delta contrib's file-planning boundary explicit (#5946)
  • Area labels: area:scan
  • Rationale: turns an implicit contract into an explicit one with regression tests; no wrong partitions are reported.
• Reuse the common native scan builder for ordinary columns in the Delta deletion-vector path (#5945)
  • Area labels: area:scan
  • Rationale: de-duplication so a later serializer fix reaches both paths — a maintainability improvement.
• Share the native-reader admission checks between the core Parquet scan and the Delta contrib (#5944)
  • Area labels: area:scan
  • Rationale: the three checks agree today; the concern is that a future fix to any of them would need a separate Delta update.
• Native Delta scan: derive the task injection key from a stable scan identity (#5943)
  • Area labels: area:scan
  • Rationale: the reporter states that exchange boundaries isolate the common self-join shapes today, making this a key-contract hardening rather than an observed wrong answer; worth revisiting as a bug if a reachable collision is found.
• Remove the legacy ShuffleWriter.output_data_file field from the native shuffle plan (#5927)
  • Area labels: area:shuffle
  • Rationale: removes a proto field kept for native binaries the versioning policy no longer allows; dead-code cleanup.
• Track whether a bug is a regression from the most recent release (#5925)
  • Area labels: none
  • Rationale: a triage-process improvement. The regression label it proposes now exists in the repository; the guide section and the bug-triage skill step do not, which is the remaining work.

Escalations to consider

• SUM(decimal) returns NULL or throws under ANSI when an intermediate sum overflows the buffer precision but the final result fits (#6002)
  • Escalated priority:high to priority:critical in this pass. In legacy mode Comet returns NULL where Spark returns 0.6, with the aggregate running entirely in Comet and no mixed plan required. The guide's first prioritization principle is "correctness over crashes", and step 1 of the decision tree puts a silent wrong result at critical regardless of whether the ANSI variant raises.
• Native S3 scan on EKS/IRSA turns a transient STS throttle into a hard 403 storm (#6024)
  • Left at priority:medium, but it matches the guide's trigger "a priority:medium bug ... affects a common workload → consider escalating to priority:high". EKS with IRSA against S3-backed tables is a mainstream deployment, and the reported incident lost 238 of 250 executors at once on a real run.
• ci: a transient Maven Central failure in lint-java is reported as a scalafix violation (#6060)
  • Left at priority:low. The guide's trigger is "a priority:low CI flake is blocking PR merges consistently → escalate to priority:medium". One occurrence is documented, so the trigger is not met; if the Lint Java matrix is seen failing this way again, it should move to medium, since the fix is already modelled by the adjacent scalafix-syntactic job.

Skipped — needs more info

All seven are tracking or record issues rather than bug reports or feature requests, so no bug/enhancement label fits and requires-triage was left in place. Six of them are the output of this same process. They will reappear in every future pass until they are closed or the queue query excludes them; that is worth fixing, either by closing them once reviewed or by having the skill filter titles beginning "Bug triage results:".

• Bug triage results: 2026-09-14 (#5922)
  • The summary issue from the previous pass, awaiting reviewer sign-off. requires-triage was applied by the auto-labeller, not by a reporter.
• Backport candidates for 1.0.x: triage of every PR merged since branch-1.0 was cut (#5815)
  • A release-management triage record, described by its author as "a triage record, not a decision". It is neither a defect nor a feature request, and it was skipped on the same grounds in the 2026-09-14 pass.
• Bug triage results: 2026-08-31 (#5601)
  • Prior summary issue, open for three weeks.
• Bug triage results: 2026-08-24 (#5454)
  • Prior summary issue. Note that this one reports that no labels were applied in that pass because the token lacked Issues write permission, so its recommendations may still be unapplied; it needs more than a sign-off.
• Bug triage results: 2026-08-17 (#5385)
  • Prior summary issue, open for five weeks.
• Bug triage results: 2026-08-11 (#5325)
  • Prior summary issue, open for six weeks.
• Bug triage results: 2026-08-03 (#5231)
  • Prior summary issue, open for seven weeks.