Skip to content
Snippets Groups Projects
Commit a609eb20 authored by Wenchen Fan's avatar Wenchen Fan Committed by Davies Liu
Browse files

[SPARK-10934] [SQL] handle hashCode of unsafe array correctly

`Murmur3_x86_32.hashUnsafeWords` only accepts word-aligned bytes, but unsafe array is not.

Author: Wenchen Fan <cloud0fan@163.com>

Closes #8987 from cloud-fan/hash.
parent c4871369
No related branches found
No related tags found
No related merge requests found
......@@ -285,7 +285,11 @@ public class UnsafeArrayData extends ArrayData {
@Override
public int hashCode() {
return Murmur3_x86_32.hashUnsafeWords(baseObject, baseOffset, sizeInBytes, 42);
int result = 37;
for (int i = 0; i < sizeInBytes; i++) {
result = 37 * result + Platform.getByte(baseObject, baseOffset + i);
}
return result;
}
@Override
......
......@@ -131,4 +131,11 @@ class UnsafeRowSuite extends SparkFunSuite {
assert(emptyRow.getInt(0) === unsafeRow.getInt(0))
assert(emptyRow.getUTF8String(1) === unsafeRow.getUTF8String(1))
}
test("calling hashCode on unsafe array returned by getArray(ordinal)") {
val row = InternalRow.apply(new GenericArrayData(Array(1L)))
val unsafeRow = UnsafeProjection.create(Array[DataType](ArrayType(LongType))).apply(row)
// Makes sure hashCode on unsafe array won't crash
unsafeRow.getArray(0).hashCode()
}
}
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment