-
- Downloads
[SPARK-16400][SQL] Remove InSet filter pushdown from Parquet
## What changes were proposed in this pull request? This patch removes InSet filter pushdown from Parquet data source, since row-based pushdown is not beneficial to Spark and brings extra complexity to the code base. ## How was this patch tested? N/A Author: Reynold Xin <rxin@databricks.com> Closes #14076 from rxin/SPARK-16400.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala 6 additions, 1 deletion...rc/main/scala/org/apache/spark/sql/types/StructType.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala 12 additions, 45 deletions...rk/sql/execution/datasources/parquet/ParquetFilters.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala 0 additions, 30 deletions...ql/execution/datasources/parquet/ParquetFilterSuite.scala
Loading
Please register or sign in to comment