-
- Downloads
[SPARK-15351][SQL] RowEncoder should support array as the external type for ArrayType
## What changes were proposed in this pull request? This PR improves `RowEncoder` and `MapObjects`, to support array as the external type for `ArrayType`. The idea is straightforward, we use `Object` as the external input type for `ArrayType`, and determine its type at runtime in `MapObjects`. ## How was this patch tested? new test in `RowEncoderSuite` Author: Wenchen Fan <wenchen@databricks.com> Closes #13138 from cloud-fan/map-object.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala 2 additions, 2 deletionssql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/RowEncoder.scala 22 additions, 0 deletions...a/org/apache/spark/sql/catalyst/encoders/RowEncoder.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala 46 additions, 53 deletions...ache/spark/sql/catalyst/expressions/objects/objects.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/GenericArrayData.scala 5 additions, 0 deletions...org/apache/spark/sql/catalyst/util/GenericArrayData.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala 17 additions, 0 deletions.../apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala
Loading
Please register or sign in to comment