-
- Downloads
[SPARK-21462][SS] Added batchId to StreamingQueryProgress.json
## What changes were proposed in this pull request? - Added batchId to StreamingQueryProgress.json as that was missing from the generated json. - Also, removed recently added numPartitions from StatefulOperatorProgress as this value does not change through the query run, and there are other ways to find that. ## How was this patch tested? Updated unit tests Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #18675 from tdas/SPARK-21462.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala 1 addition, 2 deletions...che/spark/sql/execution/streaming/statefulOperators.scala
- sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala 4 additions, 5 deletions.../main/scala/org/apache/spark/sql/streaming/progress.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryStatusAndProgressSuite.scala 6 additions, 6 deletions.../sql/streaming/StreamingQueryStatusAndProgressSuite.scala
Loading
Please register or sign in to comment