Skip to content
Snippets Groups Projects
Commit c686b7dd authored by Takuya UESHIN's avatar Takuya UESHIN Committed by Michael Armbrust
Browse files

[SPARK-2968][SQL] Fix nullabilities of Explode.

Output nullabilities of `Explode` could be detemined by `ArrayType.containsNull` or `MapType.valueContainsNull`.

Author: Takuya UESHIN <ueshin@happy-camper.st>

Closes #1888 from ueshin/issues/SPARK-2968 and squashes the following commits:

d128c95 [Takuya UESHIN] Fix nullability of Explode.
parent c9c89c31
No related branches found
No related tags found
No related merge requests found
......@@ -86,19 +86,19 @@ case class Explode(attributeNames: Seq[String], child: Expression)
(child.dataType.isInstanceOf[ArrayType] || child.dataType.isInstanceOf[MapType])
private lazy val elementTypes = child.dataType match {
case ArrayType(et, _) => et :: Nil
case MapType(kt,vt, _) => kt :: vt :: Nil
case ArrayType(et, containsNull) => (et, containsNull) :: Nil
case MapType(kt, vt, valueContainsNull) => (kt, false) :: (vt, valueContainsNull) :: Nil
}
// TODO: Move this pattern into Generator.
protected def makeOutput() =
if (attributeNames.size == elementTypes.size) {
attributeNames.zip(elementTypes).map {
case (n, t) => AttributeReference(n, t, nullable = true)()
case (n, (t, nullable)) => AttributeReference(n, t, nullable)()
}
} else {
elementTypes.zipWithIndex.map {
case (t, i) => AttributeReference(s"c_$i", t, nullable = true)()
case ((t, nullable), i) => AttributeReference(s"c_$i", t, nullable)()
}
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment