-
- Downloads
[SPARK-21546][SS] dropDuplicates should ignore watermark when it's not a key
## What changes were proposed in this pull request? When the watermark is not a column of `dropDuplicates`, right now it will crash. This PR fixed this issue. ## How was this patch tested? The new unit test. Author: Shixiong Zhu <shixiong@databricks.com> Closes #18822 from zsxwing/SPARK-21546.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala 7 additions, 2 deletions...che/spark/sql/execution/streaming/statefulOperators.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/DeduplicateSuite.scala 13 additions, 0 deletions...ala/org/apache/spark/sql/streaming/DeduplicateSuite.scala
Loading
Please register or sign in to comment