-
- Downloads
[SPARK-14812][ML][MLLIB][PYTHON] Experimental, DeveloperApi annotation audit for ML
## What changes were proposed in this pull request? General decisions to follow, except where noted: * spark.mllib, pyspark.mllib: Remove all Experimental annotations. Leave DeveloperApi annotations alone. * spark.ml, pyspark.ml ** Annotate Estimator-Model pairs of classes and companion objects the same way. ** For all algorithms marked Experimental with Since tag <= 1.6, remove Experimental annotation. ** For all algorithms marked Experimental with Since tag = 2.0, leave Experimental annotation. * DeveloperApi annotations are left alone, except where noted. * No changes to which types are sealed. Exceptions where I am leaving items Experimental in spark.ml, pyspark.ml, mainly because the items are new: * Model Summary classes * MLWriter, MLReader, MLWritable, MLReadable * Evaluator and subclasses: There is discussion of changes around evaluating multiple metrics at once for efficiency. * RFormula: Its behavior may need to change slightly to match R in edge cases. * AFTSurvivalRegression * MultilayerPerceptronClassifier DeveloperApi changes: * ml.tree.Node, ml.tree.Split, and subclasses should no longer be DeveloperApi ## How was this patch tested? N/A Note to reviewers: * spark.ml.clustering.LDA underwent significant changes (additional methods), so let me know if you want me to leave it Experimental. * Be careful to check for cases where a class should no longer be Experimental but has an Experimental method, val, or other feature. I did not find such cases, but please verify. Author: Joseph K. Bradley <joseph@databricks.com> Closes #14147 from jkbradley/experimental-audit.
Showing
- mllib/src/main/scala/org/apache/spark/ml/Pipeline.scala 1 addition, 5 deletionsmllib/src/main/scala/org/apache/spark/ml/Pipeline.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala 1 addition, 6 deletions...ache/spark/ml/classification/DecisionTreeClassifier.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala 1 addition, 6 deletions...la/org/apache/spark/ml/classification/GBTClassifier.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala 0 additions, 4 deletions...g/apache/spark/ml/classification/LogisticRegression.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/NaiveBayes.scala 1 addition, 5 deletions...scala/org/apache/spark/ml/classification/NaiveBayes.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala 1 addition, 6 deletions.../scala/org/apache/spark/ml/classification/OneVsRest.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/RandomForestClassifier.scala 1 addition, 6 deletions...ache/spark/ml/classification/RandomForestClassifier.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/Binarizer.scala 1 addition, 3 deletions...rc/main/scala/org/apache/spark/ml/feature/Binarizer.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala 1 addition, 3 deletions...c/main/scala/org/apache/spark/ml/feature/Bucketizer.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala 1 addition, 5 deletions...ain/scala/org/apache/spark/ml/feature/ChiSqSelector.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/CountVectorizer.scala 1 addition, 5 deletions...n/scala/org/apache/spark/ml/feature/CountVectorizer.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/DCT.scala 1 addition, 3 deletionsmllib/src/main/scala/org/apache/spark/ml/feature/DCT.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/ElementwiseProduct.scala 1 addition, 3 deletions...cala/org/apache/spark/ml/feature/ElementwiseProduct.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/HashingTF.scala 1 addition, 3 deletions...rc/main/scala/org/apache/spark/ml/feature/HashingTF.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/IDF.scala 1 addition, 5 deletionsmllib/src/main/scala/org/apache/spark/ml/feature/IDF.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/Interaction.scala 1 addition, 3 deletions.../main/scala/org/apache/spark/ml/feature/Interaction.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/LabeledPoint.scala 2 additions, 0 deletions...main/scala/org/apache/spark/ml/feature/LabeledPoint.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/MinMaxScaler.scala 1 addition, 5 deletions...main/scala/org/apache/spark/ml/feature/MinMaxScaler.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/NGram.scala 1 addition, 3 deletionsmllib/src/main/scala/org/apache/spark/ml/feature/NGram.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/Normalizer.scala 1 addition, 3 deletions...c/main/scala/org/apache/spark/ml/feature/Normalizer.scala
Loading
Please register or sign in to comment