-
- Downloads
[SPARK-20441][SPARK-20432][SS] Within the same streaming query, one...
[SPARK-20441][SPARK-20432][SS] Within the same streaming query, one StreamingRelation should only be transformed to one StreamingExecutionRelation ## What changes were proposed in this pull request? Within the same streaming query, when one `StreamingRelation` is referred multiple times – e.g. `df.union(df)` – we should transform it only to one `StreamingExecutionRelation`, instead of two or more different `StreamingExecutionRelation`s (each of which would have a separate set of source, source logs, ...). ## How was this patch tested? Added two test cases, each of which would fail without this patch. Author: Liwei Lin <lwlin7@gmail.com> Closes #17735 from lw-lin/SPARK-20441.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala 12 additions, 8 deletions...pache/spark/sql/execution/streaming/StreamExecution.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala 48 additions, 0 deletions...st/scala/org/apache/spark/sql/streaming/StreamSuite.scala
Loading
Please register or sign in to comment