-
- Downloads
[SPARK-6910] [SQL] Support for pushing predicates down to metastore for partition pruning
This PR forks PR #7421 authored by piaozhexiu and adds [a workaround] [1] for fixing the occasional test failures occurred in PR #7421. Please refer to these [two] [2] [comments] [3] for details. [1]: https://github.com/liancheng/spark/commit/536ac41a7e6b2abeb1f6ec1a6491bbf09ed3e591 [2]: https://github.com/apache/spark/pull/7421#issuecomment-122527391 [3]: https://github.com/apache/spark/pull/7421#issuecomment-122528059 Author: Cheolsoo Park <cheolsoop@netflix.com> Author: Cheng Lian <lian@databricks.com> Author: Michael Armbrust <michael@databricks.com> Closes #7492 from liancheng/pr-7421-workaround and squashes the following commits: 5599cc4 [Cheolsoo Park] Predicate pushdown to hive metastore 536ac41 [Cheng Lian] Sets hive.metastore.integral.jdo.pushdown to true to workaround test failures caused by in #7421
Showing
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala 31 additions, 27 deletions...cala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala 1 addition, 0 deletions...e/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala 2 additions, 2 deletions...main/scala/org/apache/spark/sql/hive/HiveStrategies.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/client/ClientInterface.scala 10 additions, 1 deletion...la/org/apache/spark/sql/hive/client/ClientInterface.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/client/ClientWrapper.scala 12 additions, 9 deletions...cala/org/apache/spark/sql/hive/client/ClientWrapper.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala 70 additions, 1 deletion...ain/scala/org/apache/spark/sql/hive/client/HiveShim.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScan.scala 4 additions, 3 deletions...a/org/apache/spark/sql/hive/execution/HiveTableScan.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/test/TestHive.scala 4 additions, 1 deletion.../main/scala/org/apache/spark/sql/hive/test/TestHive.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/client/FiltersSuite.scala 78 additions, 0 deletions...scala/org/apache/spark/sql/hive/client/FiltersSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala 8 additions, 0 deletions...cala/org/apache/spark/sql/hive/client/VersionsSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/PruningSuite.scala 1 addition, 1 deletion...la/org/apache/spark/sql/hive/execution/PruningSuite.scala
Loading
Please register or sign in to comment