-
- Downloads
[SPARK-18894][SS] Fix event time watermark delay threshold specified in months or years
## What changes were proposed in this pull request? Two changes - Fix how delays specified in months and years are translated to milliseconds - Following up on #16258, not show watermark when there is no watermarking in the query ## How was this patch tested? Updated and new unit tests Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #16304 from tdas/SPARK-18834-1. (cherry picked from commit 607a1e63) Signed-off-by:Shixiong Zhu <shixiong@databricks.com>
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/EventTimeWatermarkExec.scala 6 additions, 1 deletion...park/sql/execution/streaming/EventTimeWatermarkExec.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala 5 additions, 2 deletions...ache/spark/sql/execution/streaming/ProgressReporter.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala 1 addition, 1 deletion...pache/spark/sql/execution/streaming/StreamExecution.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/EventTimeWatermarkSuite.scala 61 additions, 14 deletions.../apache/spark/sql/streaming/EventTimeWatermarkSuite.scala
Please register or sign in to comment