-
- Downloads
[SPARK-14451][SQL] Move encoder definition into Aggregator interface
## What changes were proposed in this pull request? When we first introduced Aggregators, we required the user of Aggregators to (implicitly) specify the encoders. It would actually make more sense to have the encoders be specified by the implementation of Aggregators, since each implementation should have the most state about how to encode its own data type. Note that this simplifies the Java API because Java users no longer need to explicitly specify encoders for aggregators. ## How was this patch tested? Updated unit tests. Author: Reynold Xin <rxin@databricks.com> Closes #12231 from rxin/SPARK-14451.
Showing
- project/MimaExcludes.scala 5 additions, 0 deletionsproject/MimaExcludes.scala
- repl/scala-2.10/src/test/scala/org/apache/spark/repl/ReplSuite.scala 3 additions, 25 deletions...2.10/src/test/scala/org/apache/spark/repl/ReplSuite.scala
- repl/scala-2.11/src/test/scala/org/apache/spark/repl/ReplSuite.scala 3 additions, 26 deletions...2.11/src/test/scala/org/apache/spark/repl/ReplSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/typedaggregators.scala 23 additions, 24 deletions...ache/spark/sql/execution/aggregate/typedaggregators.scala
- sql/core/src/main/scala/org/apache/spark/sql/expressions/Aggregator.scala 26 additions, 13 deletions...n/scala/org/apache/spark/sql/expressions/Aggregator.scala
- sql/core/src/test/java/test/org/apache/spark/sql/sources/JavaDatasetAggregatorSuite.java 13 additions, 4 deletions.../apache/spark/sql/sources/JavaDatasetAggregatorSuite.java
- sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala 40 additions, 35 deletions...t/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala
Loading
Please register or sign in to comment