-
- Downloads
[SPARK-5957][ML] better handling of parameters
The design doc was posted on the JIRA page. Python changes will be in a follow-up PR. jkbradley 1. Use codegen for shared params. 1. Move shared params to package `ml.param.shared`. 1. Set default values in `Params` instead of in `Param`. 1. Add a few methods to `Params` and `ParamMap`. 1. Move schema handling to `SchemaUtils` from `Params`. - [x] check visibility of the methods added Author: Xiangrui Meng <meng@databricks.com> Closes #5431 from mengxr/SPARK-5957 and squashes the following commits: d19236d [Xiangrui Meng] fix test 26ae2d7 [Xiangrui Meng] re-gen code and mark clear protected 38b78c7 [Xiangrui Meng] update Param.toString and remove Params.explain() 409e2d5 [Xiangrui Meng] address comments 2d637bd [Xiangrui Meng] Merge remote-tracking branch 'apache/master' into SPARK-5957 eec2264 [Xiangrui Meng] make get* public in Params 4090d95 [Xiangrui Meng] Merge remote-tracking branch 'apache/master' into SPARK-5957 4fee9e7 [Xiangrui Meng] re-gen shared params 2737c2d [Xiangrui Meng] rename SharedParamCodeGen to SharedParamsCodeGen e938f81 [Xiangrui Meng] update code to set default parameter values 28ed322 [Xiangrui Meng] merge master 55be1f3 [Xiangrui Meng] merge master d63b5cc [Xiangrui Meng] fix examples 29b004c [Xiangrui Meng] update ParamsSuite 94fd98e [Xiangrui Meng] fix explain params 48d0e84 [Xiangrui Meng] add remove and update explainParams 4ac6348 [Xiangrui Meng] move schema utils to SchemaUtils add a few methods to Params 0d9594e [Xiangrui Meng] add getOrElse to ParamMap eeeffe8 [Xiangrui Meng] map ++ paramMap => extractValues 0d3fc5b [Xiangrui Meng] setDefault after param a9dbf59 [Xiangrui Meng] minor updates d9302b8 [Xiangrui Meng] generate default values 1c72579 [Xiangrui Meng] pass test compile abb7a3b [Xiangrui Meng] update default values handling dcab97a [Xiangrui Meng] add codegen for shared params
Showing
- examples/src/main/java/org/apache/spark/examples/ml/JavaDeveloperApiExample.java 2 additions, 2 deletions...org/apache/spark/examples/ml/JavaDeveloperApiExample.java
- examples/src/main/scala/org/apache/spark/examples/ml/DeveloperApiExample.scala 3 additions, 3 deletions...la/org/apache/spark/examples/ml/DeveloperApiExample.scala
- mllib/src/main/scala/org/apache/spark/ml/Estimator.scala 1 addition, 1 deletionmllib/src/main/scala/org/apache/spark/ml/Estimator.scala
- mllib/src/main/scala/org/apache/spark/ml/Pipeline.scala 5 additions, 5 deletionsmllib/src/main/scala/org/apache/spark/ml/Pipeline.scala
- mllib/src/main/scala/org/apache/spark/ml/Transformer.scala 3 additions, 2 deletionsmllib/src/main/scala/org/apache/spark/ml/Transformer.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/Classifier.scala 9 additions, 8 deletions...scala/org/apache/spark/ml/classification/Classifier.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala 7 additions, 11 deletions...g/apache/spark/ml/classification/LogisticRegression.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/ProbabilisticClassifier.scala 6 additions, 5 deletions...che/spark/ml/classification/ProbabilisticClassifier.scala
- mllib/src/main/scala/org/apache/spark/ml/evaluation/BinaryClassificationEvaluator.scala 9 additions, 6 deletions...e/spark/ml/evaluation/BinaryClassificationEvaluator.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/HashingTF.scala 4 additions, 2 deletions...rc/main/scala/org/apache/spark/ml/feature/HashingTF.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/Normalizer.scala 4 additions, 3 deletions...c/main/scala/org/apache/spark/ml/feature/Normalizer.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/StandardScaler.scala 5 additions, 4 deletions...in/scala/org/apache/spark/ml/feature/StandardScaler.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala 6 additions, 4 deletions...ain/scala/org/apache/spark/ml/feature/StringIndexer.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/Tokenizer.scala 8 additions, 8 deletions...rc/main/scala/org/apache/spark/ml/feature/Tokenizer.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala 4 additions, 3 deletions...n/scala/org/apache/spark/ml/feature/VectorAssembler.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/VectorIndexer.scala 14 additions, 11 deletions...ain/scala/org/apache/spark/ml/feature/VectorIndexer.scala
- mllib/src/main/scala/org/apache/spark/ml/impl/estimator/Predictor.scala 9 additions, 7 deletions.../scala/org/apache/spark/ml/impl/estimator/Predictor.scala
- mllib/src/main/scala/org/apache/spark/ml/param/params.scala 147 additions, 89 deletionsmllib/src/main/scala/org/apache/spark/ml/param/params.scala
- mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala 169 additions, 0 deletions...rg/apache/spark/ml/param/shared/SharedParamsCodeGen.scala
- mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala 259 additions, 0 deletions...scala/org/apache/spark/ml/param/shared/sharedParams.scala
Loading
Please register or sign in to comment