-
- Downloads
[SPARK-13446][SQL] Support reading data from Hive 2.0.1 metastore
### What changes were proposed in this pull request? This PR is to make Spark work with Hive 2.0's metastores. Compared with Hive 1.2, Hive 2.0's metastore has an API update due to removal of `HOLD_DDLTIME` in https://issues.apache.org/jira/browse/HIVE-12224. Based on the following Hive JIRA description, `HOLD_DDLTIME` should be removed from our internal API too. (https://github.com/apache/spark/pull/17063 was submitted for it): > This arcane feature was introduced long ago via HIVE-1394 It was broken as soon as it landed, HIVE-1442 and is thus useless. Fact that no one has fixed it since informs that its not really used by anyone. Better is to remove it so no one hits the bug of HIVE-1442 In the next PR, we will support 2.1.0 metastore, whose APIs were changed due to https://issues.apache.org/jira/browse/HIVE-12730. However, before that, we need a code cleanup for stats collection and setting. ### How was this patch tested? Added test cases to VersionsSuite.scala Author: Xiao Li <gatorsmile@gmail.com> Closes #17061 from gatorsmile/Hive2.
Showing
- sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala 1 addition, 0 deletions...ala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala 74 additions, 0 deletions...ain/scala/org/apache/spark/sql/hive/client/HiveShim.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala 1 addition, 0 deletions...g/apache/spark/sql/hive/client/IsolatedClientLoader.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala 7 additions, 1 deletion...main/scala/org/apache/spark/sql/hive/client/package.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala 9 additions, 2 deletions...apache/spark/sql/hive/execution/InsertIntoHiveTable.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientBuilder.scala 8 additions, 4 deletions.../org/apache/spark/sql/hive/client/HiveClientBuilder.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala 7 additions, 2 deletions...cala/org/apache/spark/sql/hive/client/VersionsSuite.scala
Loading
Please register or sign in to comment