Skip to content
Snippets Groups Projects
  • noelsmith's avatar
    7583681e
    [SPARK-10188] [PYSPARK] Pyspark CrossValidator with RMSE selects incorrect model · 7583681e
    noelsmith authored
    * Added isLargerBetter() method to Pyspark Evaluator to match the Scala version.
    * JavaEvaluator delegates isLargerBetter() to underlying Scala object.
    * Added check for isLargerBetter() in CrossValidator to determine whether to use argmin or argmax.
    * Added test cases for where smaller is better (RMSE) and larger is better (R-Squared).
    
    (This contribution is my original work and that I license the work to the project under Sparks' open source license)
    
    Author: noelsmith <mail@noelsmith.com>
    
    Closes #8399 from noel-smith/pyspark-rmse-xval-fix.
    7583681e
    History
    [SPARK-10188] [PYSPARK] Pyspark CrossValidator with RMSE selects incorrect model
    noelsmith authored
    * Added isLargerBetter() method to Pyspark Evaluator to match the Scala version.
    * JavaEvaluator delegates isLargerBetter() to underlying Scala object.
    * Added check for isLargerBetter() in CrossValidator to determine whether to use argmin or argmax.
    * Added test cases for where smaller is better (RMSE) and larger is better (R-Squared).
    
    (This contribution is my original work and that I license the work to the project under Sparks' open source license)
    
    Author: noelsmith <mail@noelsmith.com>
    
    Closes #8399 from noel-smith/pyspark-rmse-xval-fix.