-
- Downloads
[SPARK-5233][Streaming] Fix error replaying of WAL introduced bug
Because of lacking of `BlockAllocationEvent` in WAL recovery, the dangled event will mix into the new batch, which will lead to the wrong result. Details can be seen in [SPARK-5233](https://issues.apache.org/jira/browse/SPARK-5233). Author: jerryshao <saisai.shao@intel.com> Closes #4032 from jerryshao/SPARK-5233 and squashes the following commits: f0b0c0b [jerryshao] Further address the comments a237c75 [jerryshao] Address the comments e356258 [jerryshao] Fix bug in unit test 558bdc3 [jerryshao] Correctly replay the WAL log when recovering from failure
Showing
- examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala 1 addition, 1 deletion.../org/apache/spark/examples/streaming/KafkaWordCount.scala
- streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala 12 additions, 6 deletions...a/org/apache/spark/streaming/scheduler/JobGenerator.scala
- streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceivedBlockTracker.scala 9 additions, 3 deletions...ache/spark/streaming/scheduler/ReceivedBlockTracker.scala
- streaming/src/test/scala/org/apache/spark/streaming/ReceivedBlockTrackerSuite.scala 10 additions, 10 deletions...rg/apache/spark/streaming/ReceivedBlockTrackerSuite.scala
Loading
Please register or sign in to comment