-
- Downloads
[SQL] Various DataFrame DSL update.
1. Added foreach, foreachPartition, flatMap to DataFrame. 2. Added col() in dsl. 3. Support renaming columns in toDataFrame. 4. Support type inference on arrays (in addition to Seq). 5. Updated mllib to use the new DSL. Author: Reynold Xin <rxin@databricks.com> Closes #4260 from rxin/sql-dsl-update and squashes the following commits: 73466c1 [Reynold Xin] Fixed LogisticRegression. Also added better error message for resolve. fab3ccc [Reynold Xin] Bug fix. d31fcd2 [Reynold Xin] Style fix. 62608c4 [Reynold Xin] [SQL] Various DataFrame DSL update.
Showing
- mllib/src/main/scala/org/apache/spark/ml/Transformer.scala 1 addition, 2 deletionsmllib/src/main/scala/org/apache/spark/ml/Transformer.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala 5 additions, 7 deletions...g/apache/spark/ml/classification/LogisticRegression.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/StandardScaler.scala 1 addition, 2 deletions...in/scala/org/apache/spark/ml/feature/StandardScaler.scala
- mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala 11 additions, 24 deletions...c/main/scala/org/apache/spark/ml/recommendation/ALS.scala
- mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala 2 additions, 1 deletion...rc/main/scala/org/apache/spark/mllib/linalg/Vectors.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala 4 additions, 1 deletion...scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala 5 additions, 0 deletions.../org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/Column.scala 8 additions, 4 deletionssql/core/src/main/scala/org/apache/spark/sql/Column.scala
- sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala 43 additions, 4 deletionssql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala
- sql/core/src/main/scala/org/apache/spark/sql/api.scala 6 additions, 0 deletionssql/core/src/main/scala/org/apache/spark/sql/api.scala
- sql/core/src/main/scala/org/apache/spark/sql/api/java/dsl.java 7 additions, 0 deletions...ore/src/main/scala/org/apache/spark/sql/api/java/dsl.java
- sql/core/src/main/scala/org/apache/spark/sql/api/scala/dsl/package.scala 21 additions, 0 deletions...in/scala/org/apache/spark/sql/api/scala/dsl/package.scala
Loading
Please register or sign in to comment