[SPARK-17685][SQL] Make SortMergeJoinExec's currentVars is null when calling createJoinKey
## What changes were proposed in this pull request? The following SQL query cause `IndexOutOfBoundsException` issue when `LIMIT > 1310720`: ```sql CREATE TABLE tab1(int int, int2 int, str string); CREATE TABLE tab2(int int, int2 int, str string); INSERT INTO tab1 values(1,1,'str'); INSERT INTO tab1 values(2,2,'str'); INSERT INTO tab2 values(1,1,'str'); INSERT INTO tab2 values(2,3,'str'); SELECT count(*) FROM ( SELECT t1.int, t2.int2 FROM (SELECT * FROM tab1 LIMIT 1310721) t1 INNER JOIN (SELECT * FROM tab2 LIMIT 1310721) t2 ON (t1.int = t2.int AND t1.int2 = t2.int2) ) t; ``` This pull request fix this issue. ## How was this patch tested? unit tests Author: Yuming Wang <wgyumg@gmail.com> Closes #17920 from wangyum/SPARK-17685.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala 1 addition, 0 deletions.../apache/spark/sql/execution/joins/SortMergeJoinExec.scala
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameJoinSuite.scala 10 additions, 0 deletions.../test/scala/org/apache/spark/sql/DataFrameJoinSuite.scala
Loading
Please register or sign in to comment