-
- Downloads
[SPARK-9459] [SQL] use generated FromUnsafeProjection to do deep copy for UTF8String and struct
When accessing a column in UnsafeRow, it's good to avoid the copy, then we should do deep copy when turn the UnsafeRow into generic Row, this PR brings generated FromUnsafeProjection to do that. This PR also fix the expressions that cache the UTF8String, which should also copy it. Author: Davies Liu <davies@databricks.com> Closes #7840 from davies/avoid_copy and squashes the following commits: 230c8a1 [Davies Liu] address comment fd797c9 [Davies Liu] Merge branch 'master' of github.com:apache/spark into avoid_copy e095dd0 [Davies Liu] rollback rename 8ef5b0b [Davies Liu] copy String in Columnar 81360b8 [Davies Liu] fix class name 9aecb88 [Davies Liu] use FromUnsafeProjection to do deep copy for UTF8String and struct
Showing
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java 6 additions, 2 deletions.../org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala 20 additions, 13 deletions...rg/apache/spark/sql/catalyst/expressions/Projection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala 2 additions, 0 deletions...park/sql/catalyst/expressions/codegen/CodeGenerator.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateSafeProjection.scala 147 additions, 0 deletions...catalyst/expressions/codegen/GenerateSafeProjection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala 6 additions, 6 deletions...che/spark/sql/catalyst/expressions/stringOperations.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGenerationSuite.scala 43 additions, 2 deletions.../spark/sql/catalyst/expressions/CodeGenerationSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnType.scala 7 additions, 3 deletions...main/scala/org/apache/spark/sql/columnar/ColumnType.scala
- unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java 12 additions, 1 deletion...c/main/java/org/apache/spark/unsafe/types/UTF8String.java
Loading
Please register or sign in to comment