-
- Downloads
[SPARK-14296][SQL] whole stage codegen support for Dataset.map
## What changes were proposed in this pull request? This PR adds a new operator `MapElements` for `Dataset.map`, it's a 1-1 mapping and is easier to adapt to whole stage codegen framework. ## How was this patch tested? new test in `WholeStageCodegenSuite` Author: Wenchen Fan <wenchen@databricks.com> Closes #12087 from cloud-fan/map.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala 1 addition, 1 deletion...a/org/apache/spark/sql/catalyst/analysis/unresolved.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects.scala 25 additions, 15 deletions...a/org/apache/spark/sql/catalyst/expressions/objects.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala 9 additions, 0 deletions...a/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/object.scala 26 additions, 2 deletions.../org/apache/spark/sql/catalyst/plans/logical/object.scala
- sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala 11 additions, 11 deletionssql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala 2 additions, 0 deletions...cala/org/apache/spark/sql/execution/SparkStrategies.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegen.scala 5 additions, 6 deletions...la/org/apache/spark/sql/execution/WholeStageCodegen.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/objects.scala 68 additions, 1 deletion...c/main/scala/org/apache/spark/sql/execution/objects.scala
- sql/core/src/test/scala/org/apache/spark/sql/DatasetBenchmark.scala 86 additions, 0 deletions...rc/test/scala/org/apache/spark/sql/DatasetBenchmark.scala
- sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala 1 addition, 4 deletionssql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala 13 additions, 1 deletion...g/apache/spark/sql/execution/WholeStageCodegenSuite.scala
Loading
Please register or sign in to comment