-
- Downloads
[SPARK-19994][SQL] Wrong outputOrdering for right/full outer smj
## What changes were proposed in this pull request? For right outer join, values of the left key will be filled with nulls if it can't match the value of the right key, so `nullOrdering` of the left key can't be guaranteed. We should output right key order instead of left key order. For full outer join, neither left key nor right key guarantees `nullOrdering`. We should not output any ordering. In tests, besides adding three test cases for left/right/full outer sort merge join, this patch also reorganizes code in `PlannerSuite` by putting together tests for `Sort`, and also extracts common logic in Sort tests into a method. ## How was this patch tested? Corresponding test cases are added. Author: wangzhenhua <wangzhenhua@huawei.com> Author: Zhenhua Wang <wzh_zju@163.com> Closes #17331 from wzhfy/wrongOrdering.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala 11 additions, 1 deletion.../apache/spark/sql/execution/joins/SortMergeJoinExec.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala 135 additions, 98 deletions...t/scala/org/apache/spark/sql/execution/PlannerSuite.scala
Loading
Please register or sign in to comment