Complete SQL array literal support for logical types (#19338) by AnkitaAdvitot · Pull Request #19586 · apache/pinot

AnkitaAdvitot · 2026-09-17T04:30:26Z

PR flow

Flow for boolean array literal: LiteralContext type detection → ArrayLiteralTransformFunction constructor → transformToIntValuesMV using internal int array.

flowchart TD
  N0["LiteralContext#46;getPinotDataType #40;F3#41;"]:::stModified
  N1["ArrayLiteralTransformFunction constructor #40;literal#41; #40;F5#41;"]:::stModified
  N2["ArrayLiteralTransformFunction#46;transformToIntValuesMV #40;F5#41;"]:::stModified
  N0 -->|"literal context"| N1
  N1 -->|"int array"| N2
  classDef stAdded fill:#dafbe1,stroke:#1a7f37,color:#1f2328,stroke-width:2px
  classDef stModified fill:#fff8c5,stroke:#9a6700,color:#1f2328,stroke-width:2px
  classDef stRemoved fill:#ffebe9,stroke:#cf222e,color:#1f2328,stroke-width:2px
  classDef stUnchanged fill:#f6f8fa,stroke:#656d76,color:#1f2328,stroke-width:1px

AI-generated · Green: added · Yellow: modified · Red: removed · Gray: existing

Diff evidence

F3: pinot-common/src/main/java/org/apache/pinot/common/request/context/LiteralContext.java — before · after
F5: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/ArrayLiteralTransformFunction.java — before · after

Regenerate PR flow

Description

Fixes #19338.

This is a follow-up to #19247 addressing the remaining gaps for SQL array literals and logical types:

ArrayFunctions.arrayValueConstructor: Added handling for Timestamp and UUID elements so that typed Timestamp[] and UUID[] arrays are returned instead of falling back to generic Object[].
LiteralContext: Enabled multi-value array literal support for logical types BOOLEAN (boolean[] and Boolean[]), BIG_DECIMAL (BigDecimal[]), TIMESTAMP (Timestamp[]), and UUID (UUID[]).
LiteralContext.toString(): Added string formatting representations for PRIMITIVE_BOOLEAN_ARRAY, BOOLEAN_ARRAY, BIG_DECIMAL_ARRAY, TIMESTAMP_ARRAY, and UUID_ARRAY.
FunctionUtils: Registered UUID[].class mapping to ColumnDataType.UUID_ARRAY in COLUMN_DATA_TYPE_MAP.

Validation

Ran unit tests in pinot-common: LiteralContextTest, ArrayFunctionsTest, FunctionUtilsTest, LiteralSerDeTest (all 48 passed).
Verified Checkstyle with 0 violations (mvn checkstyle:check).

… types

codecov-commenter · 2026-09-25T19:43:22Z

Codecov Report

❌ Patch coverage is 90.47619% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.82%. Comparing base (3764aea) to head (03efbcc).
⚠️ Report is 58 commits behind head on master.

Files with missing lines	Patch %	Lines
...e/pinot/common/function/scalar/ArrayFunctions.java	90.00%	0 Missing and 1 partial ⚠️
...e/pinot/common/request/context/LiteralContext.java	90.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #19586      +/-   ##
============================================
- Coverage     67.82%   67.82%   -0.01%     
  Complexity     1450     1450              
============================================
  Files          3502     3502              
  Lines        226479   226494      +15     
  Branches      35790    35799       +9     
============================================
+ Hits         153613   153619       +6     
- Misses        60749    60758       +9     
  Partials      12117    12117

Flag	Coverage Δ
integration	`100.00% <ø> (ø)`
integration1	`100.00% <ø> (ø)`
integration2	`0.00% <ø> (ø)`
java-25	`67.82% <90.47%> (-0.01%)`	⬇️
lane-a	`100.00% <ø> (ø)`
lane-b	`0.00% <ø> (ø)`
temurin	`67.82% <90.47%> (-0.01%)`	⬇️
unittests	`67.82% <90.47%> (-0.01%)`	⬇️
unittests1	`58.01% <90.47%> (+0.02%)`	⬆️
unittests2	`39.53% <4.76%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Jackie-Jiang · 2026-09-25T21:31:13Z

      }
      return bytesArr;
    }
+    if (clazz == Timestamp.class) {


Could we keep the branches in the usual type order? That would put BIG_DECIMAL before BOOLEAN and TIMESTAMP before STRING and BYTES, with UUID last.

Updated the branches in arrayValueConstructor to follow the standard Pinot type order: BIG_DECIMAL, BOOLEAN, TIMESTAMP, STRING, BYTES, and UUID last. Also updated ArrayFunctionsTest accordingly.

Jackie-Jiang · 2026-09-25T21:31:13Z

      case BOOLEAN:
-        Preconditions.checkState(singleValue, "Boolean array is not supported");
-        return PinotDataType.BOOLEAN;
+        return singleValue ? PinotDataType.BOOLEAN


Could we move BOOLEAN after BIG_DECIMAL so this switch follows INT, LONG, FLOAT, DOUBLE, BIG_DECIMAL, BOOLEAN, TIMESTAMP, STRING, UUID? The separate BYTES validation can stay above the switch.

Updated LiteralContext.getPinotDataType so that the switch follows INT, LONG, FLOAT, DOUBLE, BIG_DECIMAL, BOOLEAN, TIMESTAMP, STRING, UUID, keeping the separate BYTES validation above the switch. Also aligned LiteralContext.toString() with this order.

Jackie-Jiang · 2026-09-25T21:31:13Z

        {DataType.BYTES, new byte[]{1, 2}, new byte[]{1, 2}, new byte[]{1, 3}},
-        {DataType.BYTES, new byte[][]{{1}, {2}}, new byte[][]{{1}, {2}}, new byte[][]{{1}, {3}}}
+        {DataType.BYTES, new byte[][]{{1}, {2}}, new byte[][]{{1}, {2}}, new byte[][]{{1}, {3}}},
+        {DataType.BOOLEAN, new boolean[]{true, false}, new boolean[]{true, false}, new boolean[]{true, true}},


Could we order these new cases consistently with the production type order: BIG_DECIMAL, BOOLEAN, TIMESTAMP, then the existing STRING and BYTES cases, and UUID last?

Reordered the cases in LiteralContextTest.arrayBackedValues() and the corresponding test methods to match the standard production order (BIG_DECIMAL, BOOLEAN, TIMESTAMP, STRING, BYTES, UUID), and added testTimestampLiteral for full branch coverage.

Jackie-Jiang · 2026-09-25T21:32:15Z

      case BOOLEAN:
-        Preconditions.checkState(singleValue, "Boolean array is not supported");
-        return PinotDataType.BOOLEAN;
+        return singleValue ? PinotDataType.BOOLEAN


MAJOR: Logical array literals still fail in single-stage queries. LiteralContext now accepts BOOLEAN, BIG_DECIMAL, TIMESTAMP, and UUID arrays, but TransformFunctionFactory routes array literals to ArrayLiteralTransformFunction, whose constructors support none of these types. For example, ARRAY[TRUE, FALSE] reaches its unsupported-type branch. Please extend that execution path and cover it with a query-level regression test.

Extended ArrayLiteralTransformFunction to support BOOLEAN, BIG_DECIMAL, TIMESTAMP, and UUID in both constructors (LiteralContext and List<ExpressionContext>), implemented transformToBigDecimalValuesMV, and added cross-type MV conversions. In addition, prevented CompileTimeFunctionsInvoker from folding arrayValueConstructor into unsupported Thrift literals, ensuring the logical array types are preserved and evaluated by ArrayLiteralTransformFunction. Added unit tests in ArrayLiteralTransformFunctionTest and an end-to-end query regression test in TransformQueriesTest.testArrayLiteralQueries.

Jackie-Jiang · 2026-09-25T21:32:15Z

      put(Timestamp[].class, ColumnDataType.TIMESTAMP_ARRAY);
      put(String[].class, ColumnDataType.STRING_ARRAY);
      put(byte[][].class, ColumnDataType.BYTES_ARRAY);
+      put(UUID[].class, ColumnDataType.UUID_ARRAY);


MAJOR: This maps a UUID[] scalar-function result to UUID_ARRAY, but the single-stage ScalarTransformFunctionWrapper has no transformToBytesValuesMV implementation for that result; BaseTransformFunction throws when the result is read. FunctionUtils.getRelDataType also lacks UUID_ARRAY, so planner and executor types disagree. Please complete both paths before registering UUID[] as a supported return type.

Added UUID_ARRAY in FunctionUtils.getRelDataType to return ARRAY<UUID> so planner and executor agree. In ScalarTransformFunctionWrapper, implemented transformToBytesValuesMV (supporting BYTES_ARRAY and UUID_ARRAY) as well as transformToBigDecimalValuesMV, and added UUID_ARRAY deserialization in getNonLiteralValues. Added unit tests in FunctionUtilsTest and ScalarTransformFunctionWrapperTest.

…types - Reorder type branches in ArrayFunctions.arrayValueConstructor to standard Pinot type order (BIG_DECIMAL, BOOLEAN, TIMESTAMP, STRING, BYTES, UUID) - Reorder switch branches in LiteralContext.getPinotDataType and test cases in LiteralContextTest, and add testTimestampLiteral - Extend ArrayLiteralTransformFunction to support BOOLEAN, BIG_DECIMAL, TIMESTAMP, and UUID in both constructors and MV transform conversions - Avoid folding arrayValueConstructor in CompileTimeFunctionsInvoker to preserve array literal logical types in single-stage engine - Register UUID[].class and UUID_ARRAY in FunctionUtils, and implement transformToBytesValuesMV and UUID_ARRAY conversion in ScalarTransformFunctionWrapper - Add unit and query-level regression tests across pinot-common and pinot-core

apache#19338 Complete SQL array literal support for remaining logical…

03efbcc

… types

AnkitaAdvitot mentioned this pull request Sep 17, 2026

Complete SQL array literal support for remaining logical types #19338

Open

Jackie-Jiang reviewed Sep 25, 2026

View reviewed changes

Navigation Menu

Complete SQL array literal support for logical types (#19338) - #19586

Complete SQL array literal support for logical types (#19338)#19586
AnkitaAdvitot wants to merge 2 commits into
apache:masterfrom
AnkitaAdvitot:sql-array-literal-logical-types

AnkitaAdvitot commented Sep 17, 2026 •

edited by github-actions Bot

Loading

Uh oh!

codecov-commenter commented Sep 25, 2026 •

edited

Loading

Uh oh!

Jackie-Jiang Sep 25, 2026

Uh oh!

AnkitaAdvitot Sep 28, 2026

Uh oh!

Jackie-Jiang Sep 25, 2026

Uh oh!

AnkitaAdvitot Sep 28, 2026

Uh oh!

Jackie-Jiang Sep 25, 2026

Uh oh!

AnkitaAdvitot Sep 28, 2026

Uh oh!

Jackie-Jiang Sep 25, 2026

Uh oh!

AnkitaAdvitot Sep 28, 2026

Uh oh!

Jackie-Jiang Sep 25, 2026

Uh oh!

AnkitaAdvitot Sep 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

AnkitaAdvitot commented Sep 17, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR flow

Description

Validation

Uh oh!

codecov-commenter commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

AnkitaAdvitot commented Sep 17, 2026 •

edited by github-actions Bot

Loading

codecov-commenter commented Sep 25, 2026 •

edited

Loading