-
- Downloads
[SPARK-13123][SQL] Implement whole state codegen for sort
## What changes were proposed in this pull request? This PR adds support for implementing whole state codegen for sort. Builds heaving on nongli 's PR: https://github.com/apache/spark/pull/11008 (which actually implements the feature), and adds the following changes on top: - [x] Generated code updates peak execution memory metrics - [x] Unit tests in `WholeStageCodegenSuite` and `SQLMetricsSuite` ## How was this patch tested? New unit tests in `WholeStageCodegenSuite` and `SQLMetricsSuite`. Further, all existing sort tests should pass. Author: Sameer Agarwal <sameer@databricks.com> Author: Nong Li <nong@databricks.com> Closes #11359 from sameeragarwal/sort-codegen.
Showing
- sql/catalyst/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowSorter.java 3 additions, 6 deletions...g/apache/spark/sql/execution/UnsafeExternalRowSorter.java
- sql/core/src/main/scala/org/apache/spark/sql/execution/Sort.scala 99 additions, 25 deletions.../src/main/scala/org/apache/spark/sql/execution/Sort.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegen.scala 4 additions, 4 deletions...la/org/apache/spark/sql/execution/WholeStageCodegen.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala 9 additions, 0 deletions...g/apache/spark/sql/execution/WholeStageCodegenSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala 7 additions, 0 deletions...g/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
Loading
Please register or sign in to comment