-
- Downloads
[SPARK-10049] [SPARKR] Support collecting data of ArraryType in DataFrame.
this PR : 1. Enhance reflection in RBackend. Automatically matching a Java array to Scala Seq when finding methods. Util functions like seq(), listToSeq() in R side can be removed, as they will conflict with the Serde logic that transferrs a Scala seq to R side. 2. Enhance the SerDe to support transferring a Scala seq to R side. Data of ArrayType in DataFrame after collection is observed to be of Scala Seq type. 3. Support ArrayType in createDataFrame(). Author: Sun Rui <rui.sun@intel.com> Closes #8458 from sun-rui/SPARK-10049.
Showing
- R/pkg/R/DataFrame.R 13 additions, 13 deletionsR/pkg/R/DataFrame.R
- R/pkg/R/SQLContext.R 2 additions, 2 deletionsR/pkg/R/SQLContext.R
- R/pkg/R/column.R 1 addition, 2 deletionsR/pkg/R/column.R
- R/pkg/R/functions.R 6 additions, 6 deletionsR/pkg/R/functions.R
- R/pkg/R/group.R 2 additions, 2 deletionsR/pkg/R/group.R
- R/pkg/R/schema.R 34 additions, 20 deletionsR/pkg/R/schema.R
- R/pkg/R/utils.R 0 additions, 10 deletionsR/pkg/R/utils.R
- R/pkg/inst/tests/test_sparkSQL.R 37 additions, 7 deletionsR/pkg/inst/tests/test_sparkSQL.R
- core/src/main/scala/org/apache/spark/api/r/RBackendHandler.scala 86 additions, 35 deletions...c/main/scala/org/apache/spark/api/r/RBackendHandler.scala
- core/src/main/scala/org/apache/spark/api/r/SerDe.scala 59 additions, 50 deletionscore/src/main/scala/org/apache/spark/api/r/SerDe.scala
- sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala 10 additions, 4 deletions.../src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala
Loading
Please register or sign in to comment