Skip to content
Snippets Groups Projects
Commit 73d57754 authored by Liang-Chi Hsieh's avatar Liang-Chi Hsieh Committed by Michael Armbrust
Browse files

[SPARK-6326][SQL] Improve castStruct to be faster

Current `castStruct` should be very slow. This pr slightly improves it.

Author: Liang-Chi Hsieh <viirya@gmail.com>

Closes #5017 from viirya/faster_caststruct and squashes the following commits:

385d5b0 [Liang-Chi Hsieh] Further improved.
746fcfb [Liang-Chi Hsieh] Make castStruct faster.
parent e6d1406a
No related branches found
No related tags found
No related merge requests found
......@@ -394,10 +394,17 @@ case class Cast(child: Expression, dataType: DataType) extends UnaryExpression w
val casts = from.fields.zip(to.fields).map {
case (fromField, toField) => cast(fromField.dataType, toField.dataType)
}
// TODO: This is very slow!
buildCast[Row](_, row => Row(row.toSeq.zip(casts).map {
case (v, cast) => if (v == null) null else cast(v)
}: _*))
// TODO: Could be faster?
val newRow = new GenericMutableRow(from.fields.size)
buildCast[Row](_, row => {
var i = 0
while (i < row.length) {
val v = row(i)
newRow.update(i, if (v == null) null else casts(i)(v))
i += 1
}
newRow.copy()
})
}
private[this] def cast(from: DataType, to: DataType): Any => Any = to match {
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment