-
- Downloads
[SPARK-13043][SQL] Implement remaining catalyst types in ColumnarBatch.
This includes: float, boolean, short, decimal and calendar interval. Decimal is mapped to long or byte array depending on the size and calendar interval is mapped to a struct of int and long. The only remaining type is map. The schema mapping is straightforward but we might want to revisit how we deal with this in the rest of the execution engine. Author: Nong Li <nong@databricks.com> Closes #10961 from nongli/spark-13043.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/types/DecimalType.scala 22 additions, 0 deletions...c/main/scala/org/apache/spark/sql/types/DecimalType.scala
- sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVector.java 173 additions, 7 deletions...g/apache/spark/sql/execution/vectorized/ColumnVector.java
- sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVectorUtils.java 31 additions, 3 deletions...che/spark/sql/execution/vectorized/ColumnVectorUtils.java
- sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java 25 additions, 21 deletions.../apache/spark/sql/execution/vectorized/ColumnarBatch.java
- sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OffHeapColumnVector.java 94 additions, 4 deletions...e/spark/sql/execution/vectorized/OffHeapColumnVector.java
- sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OnHeapColumnVector.java 92 additions, 2 deletions...he/spark/sql/execution/vectorized/OnHeapColumnVector.java
- sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala 39 additions, 5 deletions...e/spark/sql/execution/vectorized/ColumnarBatchSuite.scala
- unsafe/src/main/java/org/apache/spark/unsafe/Platform.java 8 additions, 0 deletionsunsafe/src/main/java/org/apache/spark/unsafe/Platform.java
Loading
Please register or sign in to comment