-
- Downloads
[SPARK-18459][SPARK-18460][STRUCTUREDSTREAMING] Rename triggerId to batchId...
[SPARK-18459][SPARK-18460][STRUCTUREDSTREAMING] Rename triggerId to batchId and add triggerDetails to json in StreamingQueryStatus ## What changes were proposed in this pull request? SPARK-18459: triggerId seems like a number that should be increasing with each trigger, whether or not there is data in it. However, actually, triggerId increases only where there is a batch of data in a trigger. So its better to rename it to batchId. SPARK-18460: triggerDetails was missing from json representation. Fixed it. ## How was this patch tested? Updated existing unit tests. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #15895 from tdas/SPARK-18459.
Showing
- python/pyspark/sql/streaming.py 3 additions, 3 deletionspython/pyspark/sql/streaming.py
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamMetrics.scala 4 additions, 4 deletions.../apache/spark/sql/execution/streaming/StreamMetrics.scala
- sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryStatus.scala 2 additions, 2 deletions...org/apache/spark/sql/streaming/StreamingQueryStatus.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/StreamMetricsSuite.scala 4 additions, 4 deletions...he/spark/sql/execution/streaming/StreamMetricsSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryListenerSuite.scala 2 additions, 2 deletions...che/spark/sql/streaming/StreamingQueryListenerSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryStatusSuite.scala 20 additions, 2 deletions...pache/spark/sql/streaming/StreamingQueryStatusSuite.scala
Loading
Please register or sign in to comment