-
- Downloads
[SPARK-12060][CORE] Avoid memory copy in JavaSerializerInstance.serialize
`JavaSerializerInstance.serialize` uses `ByteArrayOutputStream.toByteArray` to get the serialized data. `ByteArrayOutputStream.toByteArray` needs to copy the content in the internal array to a new array. However, since the array will be converted to `ByteBuffer` at once, we can avoid the memory copy. This PR added `ByteBufferOutputStream` to access the protected `buf` and convert it to a `ByteBuffer` directly. Author: Shixiong Zhu <shixiong@databricks.com> Closes #10051 from zsxwing/SPARK-12060.
Showing
- core/src/main/scala/org/apache/spark/serializer/JavaSerializer.scala 3 additions, 4 deletions...in/scala/org/apache/spark/serializer/JavaSerializer.scala
- core/src/main/scala/org/apache/spark/util/ByteBufferOutputStream.scala 31 additions, 0 deletions.../scala/org/apache/spark/util/ByteBufferOutputStream.scala
Please register or sign in to comment