Skip to content
Snippets Groups Projects
Commit 2f84a686 authored by Josh Rosen's avatar Josh Rosen
Browse files

[SPARK-17618] Guard against invalid comparisons between UnsafeRow and other formats

This patch ports changes from #15185 to Spark 2.x. In that patch, a  correctness bug in Spark 1.6.x which was caused by an invalid `equals()` comparison between an `UnsafeRow` and another row of a different format. Spark 2.x is not affected by that specific correctness bug but it can still reap the error-prevention benefits of that patch's changes, which modify  ``UnsafeRow.equals()` to throw an IllegalArgumentException if it is called with an object that is not an `UnsafeRow`.

Author: Josh Rosen <joshrosen@databricks.com>

Closes #15265 from JoshRosen/SPARK-17618-master.
parent 67c73052
No related branches found
No related tags found
No related merge requests found
...@@ -31,6 +31,7 @@ import com.esotericsoftware.kryo.KryoSerializable; ...@@ -31,6 +31,7 @@ import com.esotericsoftware.kryo.KryoSerializable;
import com.esotericsoftware.kryo.io.Input; import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output; import com.esotericsoftware.kryo.io.Output;
import org.apache.spark.sql.catalyst.InternalRow;
import org.apache.spark.sql.types.*; import org.apache.spark.sql.types.*;
import org.apache.spark.unsafe.Platform; import org.apache.spark.unsafe.Platform;
import org.apache.spark.unsafe.array.ByteArrayMethods; import org.apache.spark.unsafe.array.ByteArrayMethods;
...@@ -577,8 +578,12 @@ public final class UnsafeRow extends MutableRow implements Externalizable, KryoS ...@@ -577,8 +578,12 @@ public final class UnsafeRow extends MutableRow implements Externalizable, KryoS
return (sizeInBytes == o.sizeInBytes) && return (sizeInBytes == o.sizeInBytes) &&
ByteArrayMethods.arrayEquals(baseObject, baseOffset, o.baseObject, o.baseOffset, ByteArrayMethods.arrayEquals(baseObject, baseOffset, o.baseObject, o.baseOffset,
sizeInBytes); sizeInBytes);
} else if (!(other instanceof InternalRow)) {
return false;
} else {
throw new IllegalArgumentException(
"Cannot compare UnsafeRow to " + other.getClass().getName());
} }
return false;
} }
/** /**
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment