-
- Downloads
[SPARK-6672][SQL] convert row to catalyst in createDataFrame(RDD[Row], ...)
We assume that `RDD[Row]` contains Scala types. So we need to convert them into catalyst types in createDataFrame. liancheng Author: Xiangrui Meng <meng@databricks.com> Closes #5329 from mengxr/SPARK-6672 and squashes the following commits: 2d52644 [Xiangrui Meng] set needsConversion = false in jsonRDD 06896e4 [Xiangrui Meng] add createDataFrame without conversion 4a3767b [Xiangrui Meng] convert Row to catalyst
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala 5 additions, 0 deletions...scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
- sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala 2 additions, 1 deletionsql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala
- sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala 17 additions, 3 deletions...core/src/main/scala/org/apache/spark/sql/SQLContext.scala
- sql/core/src/main/scala/org/apache/spark/sql/parquet/newParquet.scala 2 additions, 1 deletion.../main/scala/org/apache/spark/sql/parquet/newParquet.scala
- sql/core/src/main/scala/org/apache/spark/sql/sources/commands.scala 2 additions, 1 deletion...rc/main/scala/org/apache/spark/sql/sources/commands.scala
- sql/core/src/main/scala/org/apache/spark/sql/test/ExamplePointUDT.scala 1 addition, 1 deletion...ain/scala/org/apache/spark/sql/test/ExamplePointUDT.scala
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala 8 additions, 1 deletion.../src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
Loading
Please register or sign in to comment