-
- Downloads
[SPARK-18234][SS] Made update mode public
## What changes were proposed in this pull request? Made update mode public. As part of that here are the changes. - Update DatastreamWriter to accept "update" - Changed package of InternalOutputModes from o.a.s.sql to o.a.s.sql.catalyst - Added update mode state removing with watermark to StateStoreSaveExec ## How was this patch tested? Added new tests in changed modules Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #16360 from tdas/SPARK-18234.
Showing
- sql/catalyst/src/main/java/org/apache/spark/sql/streaming/OutputMode.java 11 additions, 1 deletion.../main/java/org/apache/spark/sql/streaming/OutputMode.java
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala 2 additions, 1 deletion...k/sql/catalyst/analysis/UnsupportedOperationChecker.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/streaming/InternalOutputModes.scala 1 addition, 1 deletion...he/spark/sql/catalyst/streaming/InternalOutputModes.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationsSuite.scala 1 addition, 1 deletion...rk/sql/catalyst/analysis/UnsupportedOperationsSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala 1 addition, 1 deletion...g/apache/spark/sql/execution/datasources/DataSource.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulAggregate.scala 38 additions, 23 deletions...che/spark/sql/execution/streaming/StatefulAggregate.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala 3 additions, 2 deletions...ala/org/apache/spark/sql/execution/streaming/memory.scala
- sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala 14 additions, 3 deletions...ala/org/apache/spark/sql/streaming/DataStreamWriter.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/MemorySinkSuite.scala 22 additions, 9 deletions...pache/spark/sql/execution/streaming/MemorySinkSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/EventTimeWatermarkSuite.scala 46 additions, 9 deletions.../apache/spark/sql/streaming/EventTimeWatermarkSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSinkSuite.scala 21 additions, 1 deletion.../org/apache/spark/sql/streaming/FileStreamSinkSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala 1 addition, 1 deletion...rg/apache/spark/sql/streaming/FileStreamSourceSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala 4 additions, 4 deletions...st/scala/org/apache/spark/sql/streaming/StreamSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingAggregationSuite.scala 1 addition, 1 deletion...pache/spark/sql/streaming/StreamingAggregationSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/test/DataStreamReaderWriterSuite.scala 30 additions, 8 deletions...park/sql/streaming/test/DataStreamReaderWriterSuite.scala
Loading
Please register or sign in to comment