-
- Downloads
[SPARK-19573][SQL] Make NaN/null handling consistent in approxQuantile
## What changes were proposed in this pull request? update `StatFunctions.multipleApproxQuantiles` to handle NaN/null ## How was this patch tested? existing tests and added tests Author: Zheng RuiFeng <ruifengz@foxmail.com> Closes #16971 from zhengruifeng/quantiles_nan.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproximatePercentile.scala 2 additions, 1 deletion...atalyst/expressions/aggregate/ApproximatePercentile.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/QuantileSummaries.scala 7 additions, 5 deletions...rg/apache/spark/sql/catalyst/util/QuantileSummaries.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/QuantileSummariesSuite.scala 31 additions, 15 deletions...ache/spark/sql/catalyst/util/QuantileSummariesSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala 9 additions, 12 deletions...n/scala/org/apache/spark/sql/DataFrameStatFunctions.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/stat/StatFunctions.scala 8 additions, 2 deletions...a/org/apache/spark/sql/execution/stat/StatFunctions.scala
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameStatSuite.scala 38 additions, 19 deletions.../test/scala/org/apache/spark/sql/DataFrameStatSuite.scala
Loading
Please register or sign in to comment