-
- Downloads
[SPARK-11720][SQL][ML] Handle edge cases when count = 0 or 1 for Stats function
return Double.NaN for mean/average when count == 0 for all numeric types that is converted to Double, Decimal type continue to return null. Author: JihongMa <linlin200605@gmail.com> Closes #9705 from JihongMA/SPARK-11720.
Showing
- python/pyspark/sql/dataframe.py 1 addition, 1 deletionpython/pyspark/sql/dataframe.py
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CentralMomentAgg.scala 1 addition, 1 deletion...sql/catalyst/expressions/aggregate/CentralMomentAgg.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Kurtosis.scala 5 additions, 4 deletions...e/spark/sql/catalyst/expressions/aggregate/Kurtosis.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Skewness.scala 5 additions, 4 deletions...e/spark/sql/catalyst/expressions/aggregate/Skewness.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Stddev.scala 14 additions, 4 deletions...che/spark/sql/catalyst/expressions/aggregate/Stddev.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Variance.scala 14 additions, 4 deletions...e/spark/sql/catalyst/expressions/aggregate/Variance.scala
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala 12 additions, 6 deletions.../scala/org/apache/spark/sql/DataFrameAggregateSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala 1 addition, 1 deletion.../src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
Loading
Please register or sign in to comment