-
- Downloads
[SPARK-4394][SQL] Data Sources API Improvements
This PR adds two features to the data sources API: - Support for pushing down `IN` filters - The ability for relations to optionally provide information about their `sizeInBytes`. Author: Michael Armbrust <michael@databricks.com> Closes #3260 from marmbrus/sourcesImprovements and squashes the following commits: 9a5e171 [Michael Armbrust] Use method instead of configuration directly 99c0e6b [Michael Armbrust] Add support for sizeInBytes. 416f167 [Michael Armbrust] Support for IN in data sources API. 2a04ab3 [Michael Armbrust] Simplify implementation of InSet.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala 2 additions, 2 deletions...rg/apache/spark/sql/catalyst/expressions/predicates.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala 1 addition, 1 deletion...a/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvaluationSuite.scala 7 additions, 7 deletions.../sql/catalyst/expressions/ExpressionEvaluationSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/OptimizeInSuite.scala 1 addition, 2 deletions...apache/spark/sql/catalyst/optimizer/OptimizeInSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/sources/DataSourceStrategy.scala 2 additions, 0 deletions...ala/org/apache/spark/sql/sources/DataSourceStrategy.scala
- sql/core/src/main/scala/org/apache/spark/sql/sources/LogicalRelation.scala 1 addition, 2 deletions.../scala/org/apache/spark/sql/sources/LogicalRelation.scala
- sql/core/src/main/scala/org/apache/spark/sql/sources/filters.scala 1 addition, 0 deletions...src/main/scala/org/apache/spark/sql/sources/filters.scala
- sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala 10 additions, 1 deletion.../main/scala/org/apache/spark/sql/sources/interfaces.scala
- sql/core/src/test/scala/org/apache/spark/sql/sources/FilteredScanSuite.scala 7 additions, 0 deletions...cala/org/apache/spark/sql/sources/FilteredScanSuite.scala
Loading
Please register or sign in to comment