-
- Downloads
[SPARK-20534][SQL] Make outer generate exec return empty rows
## What changes were proposed in this pull request? Generate exec does not produce `null` values if the generator for the input row is empty and the generate operates in outer mode without join. This is caused by the fact that the `join=false` code path is different from the `join=true` code path, and that the `join=false` code path did deal with outer properly. This PR addresses this issue. ## How was this patch tested? Updated `outer*` tests in `GeneratorFunctionSuite`. Author: Herman van Hovell <hvanhovell@databricks.com> Closes #17810 from hvanhovell/SPARK-20534.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala 1 addition, 2 deletions...a/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala 1 addition, 1 deletion...rk/sql/catalyst/plans/logical/basicLogicalOperators.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/GenerateExec.scala 18 additions, 15 deletions...n/scala/org/apache/spark/sql/execution/GenerateExec.scala
- sql/core/src/test/scala/org/apache/spark/sql/GeneratorFunctionSuite.scala 6 additions, 6 deletions...t/scala/org/apache/spark/sql/GeneratorFunctionSuite.scala
Loading
Please register or sign in to comment