-
- Downloads
[SPARK-15752][SQL] Optimize metadata only query that has an aggregate whose...
[SPARK-15752][SQL] Optimize metadata only query that has an aggregate whose children are deterministic project or filter operators. ## What changes were proposed in this pull request? when query only use metadata (example: partition key), it can return results based on metadata without scanning files. Hive did it in HIVE-1003. ## How was this patch tested? add unit tests Author: Lianhui Wang <lianhuiwang09@gmail.com> Author: Wenchen Fan <wenchen@databricks.com> Author: Lianhui Wang <lianhuiwang@users.noreply.github.com> Closes #13494 from lianhuiwang/metadata-only.
Showing
- docs/sql-programming-guide.md 12 additions, 0 deletionsdocs/sql-programming-guide.md
- sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala 152 additions, 0 deletions...pache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala 1 addition, 0 deletions...scala/org/apache/spark/sql/execution/SparkOptimizer.scala
- sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala 10 additions, 0 deletions...rc/main/scala/org/apache/spark/sql/internal/SQLConf.scala
- sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala 36 additions, 0 deletions...e/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuerySuite.scala 120 additions, 0 deletions.../spark/sql/execution/OptimizeMetadataOnlyQuerySuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala 89 additions, 0 deletions...a/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
Loading
Please register or sign in to comment