-
- Downloads
[SPARK-18852][SS] StreamingQuery.lastProgress should be null when recentProgress is empty
## What changes were proposed in this pull request? Right now `StreamingQuery.lastProgress` throws NoSuchElementException and it's hard to be used in Python since Python user will just see Py4jError. This PR just makes it return null instead. ## How was this patch tested? `test("lastProgress should be null when recentProgress is empty")` Author: Shixiong Zhu <shixiong@databricks.com> Closes #16273 from zsxwing/SPARK-18852.
Showing
- python/pyspark/sql/streaming.py 7 additions, 2 deletionspython/pyspark/sql/streaming.py
- python/pyspark/sql/tests.py 17 additions, 1 deletionpython/pyspark/sql/tests.py
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala 2 additions, 2 deletions...ache/spark/sql/execution/streaming/ProgressReporter.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala 3 additions, 6 deletions...ache/spark/sql/streaming/StreamingQueryManagerSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala 20 additions, 1 deletion.../org/apache/spark/sql/streaming/StreamingQuerySuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/util/BlockingSource.scala 8 additions, 2 deletions.../org/apache/spark/sql/streaming/util/BlockingSource.scala
Loading
Please register or sign in to comment