-
- Downloads
[SPARK-18721][SS] Fix ForeachSink with watermark + append
## What changes were proposed in this pull request? Right now ForeachSink creates a new physical plan, so StreamExecution cannot retrieval metrics and watermark. This PR changes ForeachSink to manually convert InternalRows to objects without creating a new plan. ## How was this patch tested? `test("foreach with watermark: append")`. Author: Shixiong Zhu <shixiong@databricks.com> Closes #16160 from zsxwing/SPARK-18721.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ForeachSink.scala 12 additions, 33 deletions...rg/apache/spark/sql/execution/streaming/ForeachSink.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/ForeachSinkSuite.scala 67 additions, 1 deletion...ache/spark/sql/execution/streaming/ForeachSinkSuite.scala
Loading
Please register or sign in to comment