-
- Downloads
[SPARK-1225, 1241] [MLLIB] Add AreaUnderCurve and BinaryClassificationMetrics
This PR implements a generic version of `AreaUnderCurve` using the `RDD.sliding` implementation from https://github.com/apache/spark/pull/136 . It also contains refactoring of https://github.com/apache/spark/pull/160 for binary classification evaluation. Author: Xiangrui Meng <meng@databricks.com> Closes #364 from mengxr/auc and squashes the following commits: a05941d [Xiangrui Meng] replace TP/FP/TN/FN by their full names 3f42e98 [Xiangrui Meng] add (0, 0), (1, 1) to roc, and (0, 1) to pr fb4b6d2 [Xiangrui Meng] rename Evaluator to Metrics and add more metrics b1b7dab [Xiangrui Meng] fix code styles 9dc3518 [Xiangrui Meng] add tests for BinaryClassificationEvaluator ca31da5 [Xiangrui Meng] remove PredictionAndResponse 3d71525 [Xiangrui Meng] move binary evalution classes to evaluation.binary 8f78958 [Xiangrui Meng] add PredictionAndResponse dda82d5 [Xiangrui Meng] add confusion matrix aa7e278 [Xiangrui Meng] add initial version of binary classification evaluator 221ebce [Xiangrui Meng] add a new test to sliding a920865 [Xiangrui Meng] Merge branch 'sliding' into auc a9b250a [Xiangrui Meng] move sliding to mllib cab9a52 [Xiangrui Meng] use last for the last element db6cb30 [Xiangrui Meng] remove unnecessary toSeq 9916202 [Xiangrui Meng] change RDD.sliding return type to RDD[Seq[T]] 284d991 [Xiangrui Meng] change SlidedRDD to SlidingRDD c1c6c22 [Xiangrui Meng] add AreaUnderCurve 65461b2 [Xiangrui Meng] Merge branch 'sliding' into auc 5ee6001 [Xiangrui Meng] add TODO d2a600d [Xiangrui Meng] add sliding to rdd
Showing
- mllib/src/main/scala/org/apache/spark/mllib/evaluation/AreaUnderCurve.scala 62 additions, 0 deletions...la/org/apache/spark/mllib/evaluation/AreaUnderCurve.scala
- mllib/src/main/scala/org/apache/spark/mllib/evaluation/binary/BinaryClassificationMetricComputers.scala 57 additions, 0 deletions...aluation/binary/BinaryClassificationMetricComputers.scala
- mllib/src/main/scala/org/apache/spark/mllib/evaluation/binary/BinaryClassificationMetrics.scala 204 additions, 0 deletions...mllib/evaluation/binary/BinaryClassificationMetrics.scala
- mllib/src/main/scala/org/apache/spark/mllib/evaluation/binary/BinaryConfusionMatrix.scala 41 additions, 0 deletions...spark/mllib/evaluation/binary/BinaryConfusionMatrix.scala
- mllib/src/main/scala/org/apache/spark/mllib/rdd/RDDFunctions.scala 53 additions, 0 deletions.../main/scala/org/apache/spark/mllib/rdd/RDDFunctions.scala
- mllib/src/main/scala/org/apache/spark/mllib/rdd/SlidingRDD.scala 104 additions, 0 deletions...rc/main/scala/org/apache/spark/mllib/rdd/SlidingRDD.scala
- mllib/src/test/scala/org/apache/spark/mllib/evaluation/AreaUnderCurveSuite.scala 46 additions, 0 deletions...g/apache/spark/mllib/evaluation/AreaUnderCurveSuite.scala
- mllib/src/test/scala/org/apache/spark/mllib/evaluation/binary/BinaryClassificationMetricsSuite.scala 55 additions, 0 deletions.../evaluation/binary/BinaryClassificationMetricsSuite.scala
- mllib/src/test/scala/org/apache/spark/mllib/rdd/RDDFunctionsSuite.scala 49 additions, 0 deletions.../scala/org/apache/spark/mllib/rdd/RDDFunctionsSuite.scala
Loading
Please register or sign in to comment