-
- Downloads
[SPARK-7026] [SQL] fix left semi join with equi key and non-equi condition
When the `condition` extracted by `ExtractEquiJoinKeys` contain join Predicate for left semi join, we can not plan it as semiJoin. Such as SELECT * FROM testData2 x LEFT SEMI JOIN testData2 y ON x.b = y.b AND x.a >= y.a + 2 Condition `x.a >= y.a + 2` can not evaluate on table `x`, so it throw errors Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes #5643 from adrian-wang/spark7026 and squashes the following commits: cc09809 [Daoyuan Wang] refactor semijoin and add plan test 575a7c8 [Daoyuan Wang] fix notserializable 27841de [Daoyuan Wang] fix rebase 10bf124 [Daoyuan Wang] fix style 72baa02 [Daoyuan Wang] fix style 8e0afca [Daoyuan Wang] merge commits for rebase
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala 4 additions, 6 deletions...cala/org/apache/spark/sql/execution/SparkStrategies.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastLeftSemiJoinHash.scala 16 additions, 26 deletions...spark/sql/execution/joins/BroadcastLeftSemiJoinHash.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashOuterJoin.scala 1 addition, 2 deletions.../org/apache/spark/sql/execution/joins/HashOuterJoin.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashSemiJoin.scala 91 additions, 0 deletions...a/org/apache/spark/sql/execution/joins/HashSemiJoin.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/LeftSemiJoinHash.scala 10 additions, 25 deletions...g/apache/spark/sql/execution/joins/LeftSemiJoinHash.scala
- sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala 12 additions, 0 deletions...e/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/joins/SemiJoinSuite.scala 74 additions, 0 deletions.../org/apache/spark/sql/execution/joins/SemiJoinSuite.scala
Loading
Please register or sign in to comment