-
- Downloads
[SPARK-13031] [SQL] cleanup codegen and improve test coverage
1. enable whole stage codegen during tests even there is only one operator supports that. 2. split doProduce() into two APIs: upstream() and doProduce() 3. generate prefix for fresh names of each operator 4. pass UnsafeRow to parent directly (avoid getters and create UnsafeRow again) 5. fix bugs and tests. Author: Davies Liu <davies@databricks.com> Closes #10944 from davies/gen_refactor.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala 11 additions, 2 deletions...park/sql/catalyst/expressions/codegen/CodeGenerator.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateMutableProjection.scala 1 addition, 1 deletion...alyst/expressions/codegen/GenerateMutableProjection.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegen.scala 131 additions, 57 deletions...la/org/apache/spark/sql/execution/WholeStageCodegen.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregate.scala 60 additions, 28 deletions...che/spark/sql/execution/aggregate/TungstenAggregate.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala 53 additions, 43 deletions...scala/org/apache/spark/sql/execution/basicOperators.scala
- sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala 53 additions, 50 deletions...e/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala 18 additions, 16 deletions...g/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala 1 addition, 1 deletion...c/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala
- sql/core/src/test/scala/org/apache/spark/sql/util/DataFrameCallbackSuite.scala 6 additions, 4 deletions...la/org/apache/spark/sql/util/DataFrameCallbackSuite.scala
Loading
Please register or sign in to comment