-
- Downloads
[SPARK-15812][SQ][STREAMING] Added support for sorting after streaming...
[SPARK-15812][SQ][STREAMING] Added support for sorting after streaming aggregation with complete mode ## What changes were proposed in this pull request? When the output mode is complete, then the output of a streaming aggregation essentially will contain the complete aggregates every time. So this is not different from a batch dataset within an incremental execution. Other non-streaming operations should be supported on this dataset. In this PR, I am just adding support for sorting, as it is a common useful functionality. Support for other operations will come later. ## How was this patch tested? Additional unit tests. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #13549 from tdas/SPARK-15812.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala 38 additions, 23 deletions...k/sql/catalyst/analysis/UnsupportedOperationChecker.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationsSuite.scala 15 additions, 2 deletions...rk/sql/catalyst/analysis/UnsupportedOperationsSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamTest.scala 17 additions, 7 deletions...est/scala/org/apache/spark/sql/streaming/StreamTest.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingAggregationSuite.scala 25 additions, 0 deletions...pache/spark/sql/streaming/StreamingAggregationSuite.scala
Loading
Please register or sign in to comment