Skip to content
Snippets Groups Projects
Commit b4d0db80 authored by zsxwing's avatar zsxwing Committed by Tathagata Das
Browse files

[SPARK-4873][Streaming] Use `Future.zip` instead of `Future.flatMap`(for-loop)...

[SPARK-4873][Streaming] Use `Future.zip` instead of `Future.flatMap`(for-loop) in WriteAheadLogBasedBlockHandler

Use `Future.zip` instead of `Future.flatMap`(for-loop). `zip` implies these two Futures will run concurrently, while `flatMap` usually means one Future depends on the other one.

Author: zsxwing <zsxwing@gmail.com>

Closes #3721 from zsxwing/SPARK-4873 and squashes the following commits:

46a2cd9 [zsxwing] Use Future.zip instead of Future.flatMap(for-loop)
parent 29fabb1b
No related branches found
No related tags found
No related merge requests found
......@@ -187,10 +187,7 @@ private[streaming] class WriteAheadLogBasedBlockHandler(
}
// Combine the futures, wait for both to complete, and return the write ahead log segment
val combinedFuture = for {
_ <- storeInBlockManagerFuture
fileSegment <- storeInWriteAheadLogFuture
} yield fileSegment
val combinedFuture = storeInBlockManagerFuture.zip(storeInWriteAheadLogFuture).map(_._2)
val segment = Await.result(combinedFuture, blockStoreTimeout)
WriteAheadLogBasedStoreResult(blockId, segment)
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment