-
- Downloads
[SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to...
[SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s https://issues.apache.org/jira/browse/SPARK-9830 This PR contains the following main changes. * Removing `AggregateExpression1`. * Removing `Aggregate` operator, which is used to evaluate `AggregateExpression1`. * Removing planner rule used to plan `Aggregate`. * Linking `MultipleDistinctRewriter` to analyzer. * Renaming `AggregateExpression2` to `AggregateExpression` and `AggregateFunction2` to `AggregateFunction`. * Updating places where we create aggregate expression. The way to create aggregate expressions is `AggregateExpression(aggregateFunction, mode, isDistinct)`. * Changing `val`s in `DeclarativeAggregate`s that touch children of this function to `lazy val`s (when we create aggregate expression in DataFrame API, children of an aggregate function can be unresolved). Author: Yin Huai <yhuai@databricks.com> Closes #9556 from yhuai/removeAgg1.
Showing
- R/pkg/R/functions.R 1 addition, 1 deletionR/pkg/R/functions.R
- python/pyspark/sql/dataframe.py 1 addition, 1 deletionpython/pyspark/sql/dataframe.py
- python/pyspark/sql/functions.py 1 addition, 1 deletionpython/pyspark/sql/functions.py
- python/pyspark/sql/tests.py 1 addition, 1 deletionpython/pyspark/sql/tests.py
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystConf.scala 9 additions, 1 deletion...in/scala/org/apache/spark/sql/catalyst/CatalystConf.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlParser.scala 9 additions, 5 deletions.../main/scala/org/apache/spark/sql/catalyst/SqlParser.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala 10 additions, 16 deletions...ala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala 36 additions, 10 deletions...rg/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/DistinctAggregationRewriter.scala 23 additions, 212 deletions...k/sql/catalyst/analysis/DistinctAggregationRewriter.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala 2 additions, 0 deletions...apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala 9 additions, 11 deletions...apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala 4 additions, 0 deletions...a/org/apache/spark/sql/catalyst/analysis/unresolved.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala 12 additions, 10 deletions...ain/scala/org/apache/spark/sql/catalyst/dsl/package.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Average.scala 15 additions, 16 deletions...he/spark/sql/catalyst/expressions/aggregate/Average.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CentralMomentAgg.scala 6 additions, 7 deletions...sql/catalyst/expressions/aggregate/CentralMomentAgg.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Corr.scala 15 additions, 0 deletions...pache/spark/sql/catalyst/expressions/aggregate/Corr.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Count.scala 22 additions, 6 deletions...ache/spark/sql/catalyst/expressions/aggregate/Count.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/First.scala 7 additions, 7 deletions...ache/spark/sql/catalyst/expressions/aggregate/First.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/HyperLogLogPlusPlus.scala 17 additions, 0 deletions.../catalyst/expressions/aggregate/HyperLogLogPlusPlus.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Kurtosis.scala 2 additions, 0 deletions...e/spark/sql/catalyst/expressions/aggregate/Kurtosis.scala
Loading
Please register or sign in to comment