[SPARK-7850][BUILD] Hive 0.12.0 profile in POM should be removed by piaozhexiu · Pull Request #6393 · apache/spark

piaozhexiu · 2015-05-25T03:44:45Z

I grep'ed hive-0.12.0 in the source code and removed all the profiles and doc references.

srowen · 2015-05-25T07:14:17Z

ok to test

srowen · 2015-05-25T07:15:05Z

I'm OK with that if this is true and the intent, but then do we need hive-0.13.1 anymore either?

rxin · 2015-05-25T07:23:48Z

I think this is the intent - but I will let @marmbrus confirm.

witgo · 2015-05-25T07:37:02Z

The code is useless ?
https://github.com/apache/spark/tree/master/sql/hive

SparkQA · 2015-05-25T09:45:17Z

Test build #33461 has finished for PR 6393 at commit d5342cd.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

chenghao-intel · 2015-05-26T02:27:36Z

We probably also need to get rid of the HiveShim code as well in hive/hive-thriftserver modules.

piaozhexiu · 2015-05-26T02:59:11Z

@chenghao-intel thank you for pointing it out. I deleted 0.12 shim code.

SparkQA · 2015-05-26T05:31:04Z

Test build #33491 has finished for PR 6393 at commit 54d8dee.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

marmbrus · 2015-05-27T01:23:33Z

@srowen, we should get rid of -Phive-0.13.1 too. The build should only ever need to support one version of hive from now on. Another part of this task (would be nice to do now, but does not have to happen in this PR) is to inline the Hive13 shim, so we don't have this extra indirection anymore SPARK-6909.

piaozhexiu · 2015-05-27T02:02:54Z

@marmbrus thanks for the confirmation. I removed -Phive-0.13.1 now.

Regarding SPARK-6909, can we do it in a separate PR? I'm happy to do it if no one is planning to do it.

SparkQA · 2015-05-27T04:33:14Z

Test build #33556 timed out for PR 6393 at commit fb429ce after a configured wait of 150m.

rxin · 2015-05-27T04:41:12Z

Jenkins, retest this please.

rxin · 2015-05-27T04:41:57Z

@piaozhexiu looks good. I will merge once tests come back positive. It'd be great if you have some cycles to help out with SPARK-6909 to move the Hive 13 stuff into the hive module.

piaozhexiu · 2015-05-27T04:44:06Z

@rxin thank you! I'll look into SPARK-6909 this week.

SparkQA · 2015-05-27T07:09:39Z

Test build #33563 has finished for PR 6393 at commit fb429ce.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

rxin · 2015-05-27T07:19:36Z

Thanks. I've merged this in master & 1.4.

I grep'ed hive-0.12.0 in the source code and removed all the profiles and doc references. Author: Cheolsoo Park <cheolsoop@netflix.com> Closes #6393 from piaozhexiu/SPARK-7850 and squashes the following commits: fb429ce [Cheolsoo Park] Remove hive-0.13.1 profile 82bf09a [Cheolsoo Park] Remove hive 0.12.0 shim code f3722da [Cheolsoo Park] Remove hive-0.12.0 profile and references from POM and build docs (cherry picked from commit 6dd6458) Signed-off-by: Reynold Xin <rxin@databricks.com>

This is a follow-up on #6393. I am removing the following files in this PR. ``` ./sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala ./sql/hive-thriftserver/v0.13.1/src/main/scala/org/apache/spark/sql/hive/thriftserver/Shim13.scala ``` Basically, I re-factored the shim code as follows- * Rewrote code directly with Hive 0.13 methods, or * Converted code into private methods, or * Extracted code into separate classes But for leftover code that didn't fit in any of these cases, I created a HiveShim object. For eg, helper functions which wrap Hive 0.13 methods to work around Hive bugs are placed here. Author: Cheolsoo Park <cheolsoop@netflix.com> Closes #6604 from piaozhexiu/SPARK-6909 and squashes the following commits: 5dccc20 [Cheolsoo Park] Remove hive shim code

I grep'ed hive-0.12.0 in the source code and removed all the profiles and doc references. Author: Cheolsoo Park <cheolsoop@netflix.com> Closes apache#6393 from piaozhexiu/SPARK-7850 and squashes the following commits: fb429ce [Cheolsoo Park] Remove hive-0.13.1 profile 82bf09a [Cheolsoo Park] Remove hive 0.12.0 shim code f3722da [Cheolsoo Park] Remove hive-0.12.0 profile and references from POM and build docs

This is a follow-up on apache#6393. I am removing the following files in this PR. ``` ./sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala ./sql/hive-thriftserver/v0.13.1/src/main/scala/org/apache/spark/sql/hive/thriftserver/Shim13.scala ``` Basically, I re-factored the shim code as follows- * Rewrote code directly with Hive 0.13 methods, or * Converted code into private methods, or * Extracted code into separate classes But for leftover code that didn't fit in any of these cases, I created a HiveShim object. For eg, helper functions which wrap Hive 0.13 methods to work around Hive bugs are placed here. Author: Cheolsoo Park <cheolsoop@netflix.com> Closes apache#6604 from piaozhexiu/SPARK-6909 and squashes the following commits: 5dccc20 [Cheolsoo Park] Remove hive shim code

I grep'ed hive-0.12.0 in the source code and removed all the profiles and doc references. Author: Cheolsoo Park <cheolsoop@netflix.com> Closes apache#6393 from piaozhexiu/SPARK-7850 and squashes the following commits: fb429ce [Cheolsoo Park] Remove hive-0.13.1 profile 82bf09a [Cheolsoo Park] Remove hive 0.12.0 shim code f3722da [Cheolsoo Park] Remove hive-0.12.0 profile and references from POM and build docs

This is a follow-up on apache#6393. I am removing the following files in this PR. ``` ./sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala ./sql/hive-thriftserver/v0.13.1/src/main/scala/org/apache/spark/sql/hive/thriftserver/Shim13.scala ``` Basically, I re-factored the shim code as follows- * Rewrote code directly with Hive 0.13 methods, or * Converted code into private methods, or * Extracted code into separate classes But for leftover code that didn't fit in any of these cases, I created a HiveShim object. For eg, helper functions which wrap Hive 0.13 methods to work around Hive bugs are placed here. Author: Cheolsoo Park <cheolsoop@netflix.com> Closes apache#6604 from piaozhexiu/SPARK-6909 and squashes the following commits: 5dccc20 [Cheolsoo Park] Remove hive shim code

piaozhexiu force-pushed the SPARK-7850 branch from d5342cd to 54d8dee Compare May 26, 2015 02:58

Cheolsoo Park added 3 commits May 26, 2015 19:00

Remove hive-0.12.0 profile and references from POM and build docs

f3722da

Remove hive 0.12.0 shim code

82bf09a

Remove hive-0.13.1 profile

fb429ce

piaozhexiu force-pushed the SPARK-7850 branch from 54d8dee to fb429ce Compare May 27, 2015 02:00

asfgit closed this in 6dd6458 May 27, 2015

piaozhexiu mentioned this pull request Jun 3, 2015

[SPARK-6909][SQL] Remove Hive Shim code #6604

Closed

Conversation

piaozhexiu commented May 25, 2015

Uh oh!

srowen commented May 25, 2015

Uh oh!

srowen commented May 25, 2015

Uh oh!

rxin commented May 25, 2015

Uh oh!

witgo commented May 25, 2015

Uh oh!

SparkQA commented May 25, 2015

Uh oh!

chenghao-intel commented May 26, 2015

Uh oh!

piaozhexiu commented May 26, 2015

Uh oh!

SparkQA commented May 26, 2015

Uh oh!

marmbrus commented May 27, 2015

Uh oh!

piaozhexiu commented May 27, 2015

Uh oh!

SparkQA commented May 27, 2015

Uh oh!

rxin commented May 27, 2015

Uh oh!

rxin commented May 27, 2015

Uh oh!

piaozhexiu commented May 27, 2015

Uh oh!

SparkQA commented May 27, 2015

Uh oh!

rxin commented May 27, 2015

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants