-
- Downloads
[SPARK-10978][SQL] Allow data sources to eliminate filters
This PR adds a new method `unhandledFilters` to `BaseRelation`. Data sources which implement this method properly may avoid the overhead of defensive filtering done by Spark SQL. Author: Cheng Lian <lian@databricks.com> Closes #9399 from liancheng/spark-10978.unhandled-filters.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala 106 additions, 25 deletions.../spark/sql/execution/datasources/DataSourceStrategy.scala
- sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala 9 additions, 0 deletions.../main/scala/org/apache/spark/sql/sources/interfaces.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala 1 addition, 1 deletion...ql/execution/datasources/parquet/ParquetFilterSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/sources/FilteredScanSuite.scala 93 additions, 36 deletions...cala/org/apache/spark/sql/sources/FilteredScanSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/sources/SimpleTextHadoopFsRelationSuite.scala 46 additions, 1 deletion...e/spark/sql/sources/SimpleTextHadoopFsRelationSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/sources/SimpleTextRelation.scala 60 additions, 5 deletions...ala/org/apache/spark/sql/sources/SimpleTextRelation.scala
Loading
Please register or sign in to comment