-
- Downloads
[SPARK-13373] [SQL] generate sort merge join
## What changes were proposed in this pull request? Generates code for SortMergeJoin. ## How was the this patch tested? Unit tests and manually tested with TPCDS Q72, which showed 70% performance improvements (from 42s to 25s), but micro benchmark only show minor improvements, it may depends the distribution of data and number of columns. Author: Davies Liu <davies@databricks.com> Closes #11248 from davies/gen_smj.
Showing
- core/src/main/scala/org/apache/spark/storage/DiskBlockObjectWriter.scala 1 addition, 0 deletions...cala/org/apache/spark/storage/DiskBlockObjectWriter.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/BufferedRowIterator.java 12 additions, 11 deletions...a/org/apache/spark/sql/execution/BufferedRowIterator.java
- sql/core/src/main/scala/org/apache/spark/sql/execution/Expand.scala 2 additions, 2 deletions...rc/main/scala/org/apache/spark/sql/execution/Expand.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegen.scala 49 additions, 22 deletions...la/org/apache/spark/sql/execution/WholeStageCodegen.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregate.scala 2 additions, 2 deletions...che/spark/sql/execution/aggregate/TungstenAggregate.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala 12 additions, 8 deletions...scala/org/apache/spark/sql/execution/basicOperators.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoin.scala 2 additions, 2 deletions.../apache/spark/sql/execution/joins/BroadcastHashJoin.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/CartesianProduct.scala 0 additions, 1 deletion...g/apache/spark/sql/execution/joins/CartesianProduct.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoin.scala 244 additions, 3 deletions.../org/apache/spark/sql/execution/joins/SortMergeJoin.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/BenchmarkWholeStageCodegen.scala 34 additions, 0 deletions...ache/spark/sql/execution/BenchmarkWholeStageCodegen.scala
- sql/hive/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala 2 additions, 1 deletion...cala/org/apache/spark/sql/sources/BucketedReadSuite.scala
Loading
Please register or sign in to comment