-
- Downloads
[SPARK-5596] [mllib] ML model import/export for GLMs, NaiveBayes
This is a PR for Parquet-based model import/export. Please see the design doc on [the JIRA](https://issues.apache.org/jira/browse/SPARK-4587). Note: This includes only a subset of regression and classification models: * NaiveBayes, SVM, LogisticRegression * LinearRegression, RidgeRegression, Lasso Follow-up PRs will cover other models. Sketch of current contents: * New traits: Saveable, Loader * Implementations for some algorithms * Also: Added LogisticRegressionModel.getThreshold method (so that unit test could check the threshold) CC: mengxr selvinsource Author: Joseph K. Bradley <joseph@databricks.com> Closes #4233 from jkbradley/ml-import-export and squashes the following commits: 87c4eb8 [Joseph K. Bradley] small cleanups 12d9059 [Joseph K. Bradley] Many cleanups after code review. Major changes: Storing numFeatures, numClasses in model metadata. Improvements to unit tests b4ee064 [Joseph K. Bradley] Reorganized save/load for regression and classification. Renamed concepts to Saveable, Loader a34aef5 [Joseph K. Bradley] Merge remote-tracking branch 'upstream/master' into ml-import-export ee99228 [Joseph K. Bradley] scala style fix 79675d5 [Joseph K. Bradley] cleanups in LogisticRegression after rebasing after multinomial PR d1e5882 [Joseph K. Bradley] organized imports 2935963 [Joseph K. Bradley] Added save/load and tests for most classification and regression models c495dba [Joseph K. Bradley] made version for model import/export local to each model 1496852 [Joseph K. Bradley] Added save/load for NaiveBayes 8d46386 [Joseph K. Bradley] Added save/load to NaiveBayes 1577d70 [Joseph K. Bradley] fixed issues after rebasing on master (DataFrame patch) 64914a3 [Joseph K. Bradley] added getThreshold to SVMModel b1fc5ec [Joseph K. Bradley] small cleanups 418ba1b [Joseph K. Bradley] Added save, load to mllib.classification.LogisticRegressionModel, plus test suite
Showing
- mllib/src/main/scala/org/apache/spark/mllib/classification/ClassificationModel.scala 20 additions, 0 deletions...ache/spark/mllib/classification/ClassificationModel.scala
- mllib/src/main/scala/org/apache/spark/mllib/classification/LogisticRegression.scala 64 additions, 3 deletions...pache/spark/mllib/classification/LogisticRegression.scala
- mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala 84 additions, 3 deletions...la/org/apache/spark/mllib/classification/NaiveBayes.scala
- mllib/src/main/scala/org/apache/spark/mllib/classification/SVM.scala 49 additions, 2 deletions...ain/scala/org/apache/spark/mllib/classification/SVM.scala
- mllib/src/main/scala/org/apache/spark/mllib/classification/impl/GLMClassificationModel.scala 95 additions, 0 deletions...rk/mllib/classification/impl/GLMClassificationModel.scala
- mllib/src/main/scala/org/apache/spark/mllib/regression/Lasso.scala 30 additions, 3 deletions.../main/scala/org/apache/spark/mllib/regression/Lasso.scala
- mllib/src/main/scala/org/apache/spark/mllib/regression/LinearRegression.scala 32 additions, 3 deletions.../org/apache/spark/mllib/regression/LinearRegression.scala
- mllib/src/main/scala/org/apache/spark/mllib/regression/RegressionModel.scala 21 additions, 1 deletion...a/org/apache/spark/mllib/regression/RegressionModel.scala
- mllib/src/main/scala/org/apache/spark/mllib/regression/RidgeRegression.scala 33 additions, 5 deletions...a/org/apache/spark/mllib/regression/RidgeRegression.scala
- mllib/src/main/scala/org/apache/spark/mllib/regression/impl/GLMRegressionModel.scala 86 additions, 0 deletions...ache/spark/mllib/regression/impl/GLMRegressionModel.scala
- mllib/src/main/scala/org/apache/spark/mllib/tree/model/DecisionTreeModel.scala 0 additions, 1 deletion...org/apache/spark/mllib/tree/model/DecisionTreeModel.scala
- mllib/src/main/scala/org/apache/spark/mllib/util/modelSaveLoad.scala 139 additions, 0 deletions...ain/scala/org/apache/spark/mllib/util/modelSaveLoad.scala
- mllib/src/test/scala/org/apache/spark/mllib/classification/LogisticRegressionSuite.scala 68 additions, 2 deletions.../spark/mllib/classification/LogisticRegressionSuite.scala
- mllib/src/test/scala/org/apache/spark/mllib/classification/NaiveBayesSuite.scala 34 additions, 6 deletions...g/apache/spark/mllib/classification/NaiveBayesSuite.scala
- mllib/src/test/scala/org/apache/spark/mllib/classification/SVMSuite.scala 36 additions, 0 deletions...cala/org/apache/spark/mllib/classification/SVMSuite.scala
- mllib/src/test/scala/org/apache/spark/mllib/regression/LassoSuite.scala 24 additions, 0 deletions.../scala/org/apache/spark/mllib/regression/LassoSuite.scala
- mllib/src/test/scala/org/apache/spark/mllib/regression/LinearRegressionSuite.scala 24 additions, 0 deletions...apache/spark/mllib/regression/LinearRegressionSuite.scala
- mllib/src/test/scala/org/apache/spark/mllib/regression/RidgeRegressionSuite.scala 24 additions, 0 deletions.../apache/spark/mllib/regression/RidgeRegressionSuite.scala
Loading
Please register or sign in to comment