-
- Downloads
[SPARK-6885] [ML] decision tree support predict class probabilities
Decision tree support predict class probabilities. Implement the prediction probabilities function referred the old DecisionTree API and the [sklean API](https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/tree/tree.py#L593). I make the DecisionTreeClassificationModel inherit from ProbabilisticClassificationModel, make the predictRaw to return the raw counts vector and make raw2probabilityInPlace/predictProbability return the probabilities for each prediction. Author: Yanbo Liang <ybliang8@gmail.com> Closes #7694 from yanboliang/spark-6885 and squashes the following commits: 08d5b7f [Yanbo Liang] fix ImpurityStats null parameters and raw2probabilityInPlace sum = 0 issue 2174278 [Yanbo Liang] solve merge conflicts 7e90ba8 [Yanbo Liang] fix typos 33ae183 [Yanbo Liang] fix annotation ff043d3 [Yanbo Liang] raw2probabilityInPlace should operate in-place c32d6ce [Yanbo Liang] optimize calculateImpurityStats function again 6167fb0 [Yanbo Liang] optimize calculateImpurityStats function fbbe2ec [Yanbo Liang] eliminate duplicated struct and code beb1634 [Yanbo Liang] try to eliminate impurityStats for each LearningNode 99e8943 [Yanbo Liang] code optimization 5ec3323 [Yanbo Liang] implement InformationGainAndImpurityStats 227c91b [Yanbo Liang] refactor LearningNode to store ImpurityCalculator d746ffc [Yanbo Liang] decision tree support predict class probabilities
Showing
- mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala 31 additions, 9 deletions...ache/spark/ml/classification/DecisionTreeClassifier.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala 1 addition, 1 deletion...la/org/apache/spark/ml/classification/GBTClassifier.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/RandomForestClassifier.scala 1 addition, 1 deletion...ache/spark/ml/classification/RandomForestClassifier.scala
- mllib/src/main/scala/org/apache/spark/ml/regression/DecisionTreeRegressor.scala 1 addition, 1 deletion...rg/apache/spark/ml/regression/DecisionTreeRegressor.scala
- mllib/src/main/scala/org/apache/spark/ml/regression/GBTRegressor.scala 1 addition, 1 deletion...n/scala/org/apache/spark/ml/regression/GBTRegressor.scala
- mllib/src/main/scala/org/apache/spark/ml/regression/RandomForestRegressor.scala 1 addition, 1 deletion...rg/apache/spark/ml/regression/RandomForestRegressor.scala
- mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala 44 additions, 36 deletionsmllib/src/main/scala/org/apache/spark/ml/tree/Node.scala
- mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala 55 additions, 71 deletions...in/scala/org/apache/spark/ml/tree/impl/RandomForest.scala
- mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Entropy.scala 1 addition, 1 deletion.../scala/org/apache/spark/mllib/tree/impurity/Entropy.scala
- mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Gini.scala 1 addition, 1 deletion...ain/scala/org/apache/spark/mllib/tree/impurity/Gini.scala
- mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Impurity.scala 1 addition, 1 deletion...scala/org/apache/spark/mllib/tree/impurity/Impurity.scala
- mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Variance.scala 1 addition, 1 deletion...scala/org/apache/spark/mllib/tree/impurity/Variance.scala
- mllib/src/main/scala/org/apache/spark/mllib/tree/model/InformationGainStats.scala 60 additions, 1 deletion.../apache/spark/mllib/tree/model/InformationGainStats.scala
- mllib/src/test/scala/org/apache/spark/ml/classification/DecisionTreeClassifierSuite.scala 28 additions, 2 deletions...spark/ml/classification/DecisionTreeClassifierSuite.scala
- mllib/src/test/scala/org/apache/spark/ml/classification/GBTClassifierSuite.scala 1 addition, 1 deletion...g/apache/spark/ml/classification/GBTClassifierSuite.scala
- mllib/src/test/scala/org/apache/spark/ml/classification/RandomForestClassifierSuite.scala 1 addition, 1 deletion...spark/ml/classification/RandomForestClassifierSuite.scala
Loading
Please register or sign in to comment