-
- Downloads
[SPARK-18261][STRUCTURED STREAMING] Add statistics to MemorySink for joining
## What changes were proposed in this pull request? Right now, there is no way to join the output of a memory sink with any table: > UnsupportedOperationException: LeafNode MemoryPlan must implement statistics This patch adds statistics to MemorySink, making joining snapshots of memory streams with tables possible. ## How was this patch tested? Added a test case. Author: Liwei Lin <lwlin7@gmail.com> Closes #15786 from lw-lin/memory-sink-stat.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala 5 additions, 1 deletion...ala/org/apache/spark/sql/execution/streaming/memory.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/MemorySinkSuite.scala 16 additions, 0 deletions...cala/org/apache/spark/sql/streaming/MemorySinkSuite.scala
Please register or sign in to comment