-
- Downloads
[SPARK-7176] [ML] Add validation functionality to Param
Main change: Added isValid field to Param. Modified all usages to use isValid when relevant. Added helper methods in ParamValidate. Also overrode Params.validate() in: * CrossValidator + model * Pipeline + model I made a few updates for the elastic net patch: * I changed "tol" to "convergenceTol" * I added some documentation This PR is Scala + Java only. Python will be in a follow-up PR. CC: mengxr Author: Joseph K. Bradley <joseph@databricks.com> Closes #5740 from jkbradley/enforce-validate and squashes the following commits: ad9c6c1 [Joseph K. Bradley] re-generated sharedParams after merging with current master 76415e8 [Joseph K. Bradley] reverted convergenceTol to tol af62f4b [Joseph K. Bradley] Removed changes to SparkBuild, python linalg. Fixed test failures. Renamed ParamValidate to ParamValidators. Removed explicit type from ParamValidators calls where possible. bb2665a [Joseph K. Bradley] merged with elastic net pr ecda302 [Joseph K. Bradley] fix rat tests, plus add a little doc 6895dfc [Joseph K. Bradley] small cleanups 069ac6d [Joseph K. Bradley] many cleanups 928fb84 [Joseph K. Bradley] Maybe done a910ac7 [Joseph K. Bradley] still workin 6d60e2e [Joseph K. Bradley] Still workin b987319 [Joseph K. Bradley] Partly done with adding checks, but blocking on adding checking functionality to Param dbc9fb2 [Joseph K. Bradley] merged with master. enforcing Params.validate
Showing
- examples/src/main/java/org/apache/spark/examples/ml/JavaDeveloperApiExample.java 8 additions, 6 deletions...org/apache/spark/examples/ml/JavaDeveloperApiExample.java
- mllib/src/main/scala/org/apache/spark/ml/Pipeline.scala 16 additions, 3 deletionsmllib/src/main/scala/org/apache/spark/ml/Pipeline.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala 4 additions, 9 deletions...la/org/apache/spark/ml/classification/GBTClassifier.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/HashingTF.scala 7 additions, 5 deletions...rc/main/scala/org/apache/spark/ml/feature/HashingTF.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/Normalizer.scala 6 additions, 5 deletions...c/main/scala/org/apache/spark/ml/feature/Normalizer.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/PolynomialExpansion.scala 6 additions, 3 deletions...ala/org/apache/spark/ml/feature/PolynomialExpansion.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/StandardScaler.scala 6 additions, 4 deletions...in/scala/org/apache/spark/ml/feature/StandardScaler.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/Tokenizer.scala 11 additions, 9 deletions...rc/main/scala/org/apache/spark/ml/feature/Tokenizer.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/VectorIndexer.scala 8 additions, 10 deletions...ain/scala/org/apache/spark/ml/feature/VectorIndexer.scala
- mllib/src/main/scala/org/apache/spark/ml/impl/tree/treeParams.scala 39 additions, 76 deletions...main/scala/org/apache/spark/ml/impl/tree/treeParams.scala
- mllib/src/main/scala/org/apache/spark/ml/param/params.scala 162 additions, 17 deletionsmllib/src/main/scala/org/apache/spark/ml/param/params.scala
- mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala 24 additions, 11 deletions...rg/apache/spark/ml/param/shared/SharedParamsCodeGen.scala
- mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala 44 additions, 78 deletions...scala/org/apache/spark/ml/param/shared/sharedParams.scala
- mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala 21 additions, 14 deletions...c/main/scala/org/apache/spark/ml/recommendation/ALS.scala
- mllib/src/main/scala/org/apache/spark/ml/regression/GBTRegressor.scala 4 additions, 9 deletions...n/scala/org/apache/spark/ml/regression/GBTRegressor.scala
- mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala 14 additions, 2 deletions...ala/org/apache/spark/ml/regression/LinearRegression.scala
- mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala 17 additions, 5 deletions...ain/scala/org/apache/spark/ml/tuning/CrossValidator.scala
- mllib/src/main/scala/org/apache/spark/mllib/tree/GradientBoostedTrees.scala 3 additions, 1 deletion...la/org/apache/spark/mllib/tree/GradientBoostedTrees.scala
- mllib/src/test/java/org/apache/spark/ml/param/JavaParamsSuite.java 66 additions, 0 deletions.../test/java/org/apache/spark/ml/param/JavaParamsSuite.java
- mllib/src/test/java/org/apache/spark/ml/param/JavaTestParams.java 63 additions, 0 deletions...c/test/java/org/apache/spark/ml/param/JavaTestParams.java
Loading
Please register or sign in to comment