Skip to content
Snippets Groups Projects
Commit 0187cef0 authored by Matei Zaharia's avatar Matei Zaharia
Browse files

Fix removal from shuffleToMapStage to search for a key-value pair with

our stage instead of using our shuffleID.
parent cd32d5e4
No related branches found
No related tags found
No related merge requests found
......@@ -381,8 +381,8 @@ class DAGScheduler(
running -= stage
}
stageToInfos -= stage
for (shuffleDep <- stage.shuffleDep) {
shuffleToMapStage.remove(shuffleDep.shuffleId)
for ((k, v) <- shuffleToMapStage.find(_._2 == stage)) {
shuffleToMapStage.remove(k)
}
if (pendingTasks.contains(stage) && !pendingTasks(stage).isEmpty) {
logDebug("Removing pending status for stage %d".format(stageId))
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment