-
- Downloads
[SPARK-9834] [MLLIB] implement weighted least squares via normal equation
The goal of this PR is to have a weighted least squares implementation that takes the normal equation approach, and hence to be able to provide R-like summary statistics and support IRLS (used by GLMs). The tests match R's lm and glmnet. There are couple TODOs that can be addressed in future PRs: * consolidate summary statistics aggregators * move `dspr` to `BLAS` * etc It would be nice to have this merged first because it blocks couple other features. dbtsai Author: Xiangrui Meng <meng@databricks.com> Closes #8588 from mengxr/SPARK-9834.
Showing
- mllib/src/main/scala/org/apache/spark/ml/optim/WeightedLeastSquares.scala 296 additions, 0 deletions...cala/org/apache/spark/ml/optim/WeightedLeastSquares.scala
- mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala 7 additions, 0 deletions...b/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala
- mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala 2 additions, 1 deletion...org/apache/spark/mllib/linalg/distributed/RowMatrix.scala
- mllib/src/test/scala/org/apache/spark/ml/optim/WeightedLeastSquaresSuite.scala 133 additions, 0 deletions...org/apache/spark/ml/optim/WeightedLeastSquaresSuite.scala
Loading
Please register or sign in to comment