-
- Downloads
[SPARK-15198][SQL] Support for pushing down filters for boolean types in ORC data source
## What changes were proposed in this pull request? It seems ORC supports all the types in ([`PredicateLeaf.Type`](https://github.com/apache/hive/blob/e085b7e9bd059d91aaf013df0db4d71dca90ec6f/storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java#L50-L56)) which includes boolean types. So, this was tested first. This PR adds the support for pushing filters down for `BooleanType` in ORC data source. This PR also removes `OrcTableScan` class and the companion object, which is not used anymore. ## How was this patch tested? Unittest in `OrcFilterSuite` and `OrcQuerySuite`. Author: hyukjinkwon <gurwls223@gmail.com> Closes #12972 from HyukjinKwon/SPARK-15198.
Showing
- sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala 4 additions, 6 deletions...n/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFilters.scala 1 addition, 1 deletion...main/scala/org/apache/spark/sql/hive/orc/OrcFilters.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcFilterSuite.scala 21 additions, 4 deletions.../scala/org/apache/spark/sql/hive/orc/OrcFilterSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcQuerySuite.scala 13 additions, 0 deletions...t/scala/org/apache/spark/sql/hive/orc/OrcQuerySuite.scala
Loading
Please register or sign in to comment