-
- Downloads
[SPARK-16046][DOCS] Aggregations in the Spark SQL programming guide
## What changes were proposed in this pull request? - A separate subsection for Aggregations under “Getting Started” in the Spark SQL programming guide. It mentions which aggregate functions are predefined and how users can create their own. - Examples of using the `UserDefinedAggregateFunction` abstract class for untyped aggregations in Java and Scala. - Examples of using the `Aggregator` abstract class for type-safe aggregations in Java and Scala. - Python is not covered. - The PR might not resolve the ticket since I do not know what exactly was planned by the author. In total, there are four new standalone examples that can be executed via `spark-submit` or `run-example`. The updated Spark SQL programming guide references to these examples and does not contain hard-coded snippets. ## How was this patch tested? The patch was tested locally by building the docs. The examples were run as well.  Author: aokolnychyi <okolnychyyanton@gmail.com> Closes #16329 from aokolnychyi/SPARK-16046. (cherry picked from commit 3fdce814) Signed-off-by:gatorsmile <gatorsmile@gmail.com>
Showing
- docs/sql-programming-guide.md 46 additions, 0 deletionsdocs/sql-programming-guide.md
- examples/src/main/java/org/apache/spark/examples/sql/JavaUserDefinedTypedAggregation.java 160 additions, 0 deletions...e/spark/examples/sql/JavaUserDefinedTypedAggregation.java
- examples/src/main/java/org/apache/spark/examples/sql/JavaUserDefinedUntypedAggregation.java 132 additions, 0 deletions...spark/examples/sql/JavaUserDefinedUntypedAggregation.java
- examples/src/main/resources/employees.json 4 additions, 0 deletionsexamples/src/main/resources/employees.json
- examples/src/main/scala/org/apache/spark/examples/sql/UserDefinedTypedAggregation.scala 91 additions, 0 deletions...ache/spark/examples/sql/UserDefinedTypedAggregation.scala
- examples/src/main/scala/org/apache/spark/examples/sql/UserDefinedUntypedAggregation.scala 100 additions, 0 deletions...he/spark/examples/sql/UserDefinedUntypedAggregation.scala
Loading
Please register or sign in to comment