-
- Downloads
[SPARK-14549][ML] Copy the Vector and Matrix classes from mllib to ml in mllib-local
## What changes were proposed in this pull request? This task will copy the Vector and Matrix classes from mllib to ml package in mllib-local jar. The UDTs and `since` annotation in ml vector and matrix will be removed from now. UDTs will be achieved by #SPARK-14487, and `since` will be replaced by /* since 1.2.0 */ The BLAS implementation will be copied, and some of the test utilities will be copies as well. Summary of changes: 1. In mllib-local/src/main/scala/org/apache/spark/**ml**/linalg/BLAS.scala - Copied from mllib/src/main/scala/org/apache/spark/**mllib**/linalg/BLAS.scala - logDebug("gemm: alpha is equal to 0 and beta is equal to 1. Returning C.") is removed in ml version. 2. In mllib-local/src/main/scala/org/apache/spark/**ml**/linalg/Matrices.scala - Copied from mllib/src/main/scala/org/apache/spark/**mllib**/linalg/Matrices.scala - `Since` was removed, and we'll use standard `/* Since /*` Java doc. Will be in another PR. - `UDT` related code was removed, and will use `SPARK-13944` https://github.com/apache/spark/pull/12259 to replace the annotation. 3. In mllib-local/src/main/scala/org/apache/spark/**ml**/linalg/Vectors.scala - Copied from mllib/src/main/scala/org/apache/spark/**mllib**/linalg/Vectors.scala - `Since` was removed. - `UDT` related code was removed. - In `def parseNumeric`, it was throwing `throw new SparkException(s"Cannot parse $other.")`, and now it's throwing `throw new IllegalArgumentException(s"Cannot parse $other.")` 4. In mllib/src/main/scala/org/apache/spark/**mllib**/linalg/Vectors.scala - For consistency with ML version of vector, `def parseNumeric` is now throwing `throw new IllegalArgumentException(s"Cannot parse $other.")` 5. mllib/src/main/scala/org/apache/spark/**mllib**/util/NumericParser.scala is moved to mllib-local/src/main/scala/org/apache/spark/**ml**/util/NumericParser.scala - All the `throw new SparkException` were replaced by `throw new IllegalArgumentException` ## How was this patch tested? unit tests Author: DB Tsai <dbt@netflix.com> Closes #12317 from dbtsai/dbtsai-ml-vector.
Showing
- mllib-local/pom.xml 4 additions, 0 deletionsmllib-local/pom.xml
- mllib-local/src/main/scala/org/apache/spark/ml/linalg/BLAS.scala 723 additions, 0 deletions...ocal/src/main/scala/org/apache/spark/ml/linalg/BLAS.scala
- mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala 1026 additions, 0 deletions.../src/main/scala/org/apache/spark/ml/linalg/Matrices.scala
- mllib-local/src/main/scala/org/apache/spark/ml/linalg/Vectors.scala 736 additions, 0 deletions...l/src/main/scala/org/apache/spark/ml/linalg/Vectors.scala
- mllib-local/src/test/scala/org/apache/spark/ml/SparkMLFunSuite.scala 10 additions, 3 deletions.../src/test/scala/org/apache/spark/ml/SparkMLFunSuite.scala
- mllib-local/src/test/scala/org/apache/spark/ml/linalg/BLASSuite.scala 408 additions, 0 deletions...src/test/scala/org/apache/spark/ml/linalg/BLASSuite.scala
- mllib-local/src/test/scala/org/apache/spark/ml/linalg/BreezeMatrixConversionSuite.scala 71 additions, 0 deletions.../apache/spark/ml/linalg/BreezeMatrixConversionSuite.scala
- mllib-local/src/test/scala/org/apache/spark/ml/linalg/BreezeVectorConversionSuite.scala 67 additions, 0 deletions.../apache/spark/ml/linalg/BreezeVectorConversionSuite.scala
- mllib-local/src/test/scala/org/apache/spark/ml/linalg/MatricesSuite.scala 511 additions, 0 deletions...test/scala/org/apache/spark/ml/linalg/MatricesSuite.scala
- mllib-local/src/test/scala/org/apache/spark/ml/linalg/VectorsSuite.scala 358 additions, 0 deletions.../test/scala/org/apache/spark/ml/linalg/VectorsSuite.scala
- mllib-local/src/test/scala/org/apache/spark/ml/util/TestingUtils.scala 236 additions, 0 deletions...rc/test/scala/org/apache/spark/ml/util/TestingUtils.scala
- mllib-local/src/test/scala/org/apache/spark/ml/util/TestingUtilsSuite.scala 187 additions, 0 deletions...st/scala/org/apache/spark/ml/util/TestingUtilsSuite.scala
- mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala 2 additions, 2 deletions...test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala
Loading
Please register or sign in to comment