-
- Downloads
[SPARK-11243][SQL] output UnsafeRow from columnar cache
This PR change InMemoryTableScan to output UnsafeRow, and optimize the unrolling and scanning by coping the bytes for var-length types between UnsafeRow and ByteBuffer directly without creating the wrapper objects. When scanning the decimals in TPC-DS store_sales table, it's 80% faster (copy it as long without create Decimal objects). Author: Davies Liu <davies@databricks.com> Closes #9203 from davies/unsafe_cache.
Showing
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java 26 additions, 5 deletions.../org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java 60 additions, 18 deletions...k/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java 72 additions, 30 deletions...ark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala 2 additions, 55 deletions...talyst/expressions/codegen/GenerateUnsafeProjection.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnType.scala 70 additions, 11 deletions...main/scala/org/apache/spark/sql/columnar/ColumnType.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/GenerateColumnAccessor.scala 58 additions, 10 deletions...rg/apache/spark/sql/columnar/GenerateColumnAccessor.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/InMemoryColumnarTableScan.scala 3 additions, 3 deletions...apache/spark/sql/columnar/InMemoryColumnarTableScan.scala
Loading
Please register or sign in to comment