Skip to content

fix: CometReader.loadVector should not overwrite dictionary ids - #476

Merged
viirya merged 2 commits into
apache:mainfrom
viirya:fix_dictionary_array_import
May 28, 2024
Merged

fix: CometReader.loadVector should not overwrite dictionary ids#476
viirya merged 2 commits into
apache:mainfrom
viirya:fix_dictionary_array_import

Conversation

@viirya

@viirya viirya commented May 26, 2024

Copy link
Copy Markdown
Member

Which issue does this PR close?

Closes #475.

Rationale for this change

What changes are included in this PR?

How are these changes tested?

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 0% with 25 lines in your changes are missing coverage. Please review.

Project coverage is 34.03%. Comparing base (9125e6a) to head (0eb2ca6).
Report is 4 commits behind head on main.

Files Patch % Lines
...scala/org/apache/arrow/c/CometSchemaImporter.scala 0.00% 9 Missing ⚠️
...in/java/org/apache/comet/parquet/ColumnReader.java 0.00% 8 Missing ⚠️
.../scala/org/apache/spark/sql/comet/util/Utils.scala 0.00% 4 Missing ⚠️
...ain/java/org/apache/comet/parquet/BatchReader.java 0.00% 2 Missing ⚠️
...ava/org/apache/comet/parquet/LazyColumnReader.java 0.00% 1 Missing ⚠️
.../src/main/java/org/apache/comet/parquet/Utils.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main     #476      +/-   ##
============================================
- Coverage     34.05%   34.03%   -0.02%     
  Complexity      859      859              
============================================
  Files           116      117       +1     
  Lines         38680    38696      +16     
  Branches       8568     8567       -1     
============================================
  Hits          13171    13171              
- Misses        22746    22762      +16     
  Partials       2763     2763              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@viirya

viirya commented May 26, 2024

Copy link
Copy Markdown
Member Author

cc @sunchao @andygrove

Comment thread common/src/main/scala/org/apache/arrow/c/CometSchemaImporter.scala Outdated
Comment thread common/src/main/scala/org/apache/spark/sql/comet/util/Utils.scala Outdated
@viirya

viirya commented May 27, 2024

Copy link
Copy Markdown
Member Author

CometSchemaImporter now is implemented in Java.

@viirya
viirya force-pushed the fix_dictionary_array_import branch from c5e4f91 to 4678304 Compare May 27, 2024 17:25
@viirya
viirya force-pushed the fix_dictionary_array_import branch from 4678304 to 0bc4933 Compare May 27, 2024 17:26
@viirya

viirya commented May 27, 2024

Copy link
Copy Markdown
Member Author

@sunchao Any more comments? Thanks.

@viirya
viirya merged commit 479a97a into apache:main May 28, 2024
@viirya

viirya commented May 28, 2024

Copy link
Copy Markdown
Member Author

Merged. Thanks @andygrove @sunchao

@viirya
viirya deleted the fix_dictionary_array_import branch May 28, 2024 01:55
himadripal pushed a commit to himadripal/datafusion-comet that referenced this pull request Sep 7, 2024
…he#476)

* fix: CometReader.loadVector should not overwrite dictionary ids

* For review

(cherry picked from commit 479a97a)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CometReader.loadVector should not overwrite dictionary ids

4 participants