Skip to content
Snippets Groups Projects
Commit c9c89c31 authored by Takuya UESHIN's avatar Takuya UESHIN Committed by Michael Armbrust
Browse files

[SPARK-2965][SQL] Fix HashOuterJoin output nullabilities.

Output attributes of opposite side of `OuterJoin` should be nullable.

Author: Takuya UESHIN <ueshin@happy-camper.st>

Closes #1887 from ueshin/issues/SPARK-2965 and squashes the following commits:

bcb2d37 [Takuya UESHIN] Fix HashOuterJoin output nullabilities.
parent 647aeba3
No related branches found
No related tags found
No related merge requests found
...@@ -168,7 +168,18 @@ case class HashOuterJoin( ...@@ -168,7 +168,18 @@ case class HashOuterJoin(
override def requiredChildDistribution = override def requiredChildDistribution =
ClusteredDistribution(leftKeys) :: ClusteredDistribution(rightKeys) :: Nil ClusteredDistribution(leftKeys) :: ClusteredDistribution(rightKeys) :: Nil
def output = left.output ++ right.output override def output = {
joinType match {
case LeftOuter =>
left.output ++ right.output.map(_.withNullability(true))
case RightOuter =>
left.output.map(_.withNullability(true)) ++ right.output
case FullOuter =>
left.output.map(_.withNullability(true)) ++ right.output.map(_.withNullability(true))
case x =>
throw new Exception(s"HashOuterJoin should not take $x as the JoinType")
}
}
// TODO we need to rewrite all of the iterators with our own implementation instead of the Scala // TODO we need to rewrite all of the iterators with our own implementation instead of the Scala
// iterator for performance purpose. // iterator for performance purpose.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment