-
- Downloads
[SPARK-10585] [SQL] only copy data once when generate unsafe projection
This PR is a completely rewritten of GenerateUnsafeProjection, to accomplish the goal of copying data only once. The old code of GenerateUnsafeProjection is still there to reduce review difficulty. Instead of creating unsafe conversion code for struct, array and map, we create code of writing the content to the global row buffer. Author: Wenchen Fan <cloud0fan@163.com> Author: Wenchen Fan <cloud0fan@outlook.com> Closes #8747 from cloud-fan/copy-once.
Showing
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java 3 additions, 4 deletions...pache/spark/sql/catalyst/expressions/UnsafeArrayData.java
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeMapData.java 10 additions, 5 deletions.../apache/spark/sql/catalyst/expressions/UnsafeMapData.java
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeReaders.java 6 additions, 0 deletions.../apache/spark/sql/catalyst/expressions/UnsafeReaders.java
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java 2 additions, 2 deletions.../org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRowWriters.java 2 additions, 2 deletions...ache/spark/sql/catalyst/expressions/UnsafeRowWriters.java
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeWriters.java 2 additions, 2 deletions.../apache/spark/sql/catalyst/expressions/UnsafeWriters.java
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java 54 additions, 0 deletions.../spark/sql/catalyst/expressions/codegen/BufferHolder.java
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java 151 additions, 0 deletions...k/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java 199 additions, 0 deletions...ark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala 1 addition, 0 deletions...park/sql/catalyst/expressions/codegen/CodeGenerator.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala 283 additions, 1 deletion...talyst/expressions/codegen/GenerateUnsafeProjection.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/UnsafeRowConverterSuite.scala 237 additions, 68 deletions...rk/sql/catalyst/expressions/UnsafeRowConverterSuite.scala
Loading
Please register or sign in to comment