Skip to content
Snippets Groups Projects
Commit 4ec5c360 authored by huangzhaowei's avatar huangzhaowei Committed by Marcelo Vanzin
Browse files

[SPARK-16868][WEB UI] Fix executor be both dead and alive on executor ui.

## What changes were proposed in this pull request?
In a heavy pressure of the spark application, since the executor will register it to driver block manager twice(because of heart beats), the executor will show as picture show:
![image](https://cloud.githubusercontent.com/assets/7404824/17467245/c1359094-5d4e-11e6-843a-f6d6347e1bf6.png)

## How was this patch tested?
NA

Details in: [SPARK-16868](https://issues.apache.org/jira/browse/SPARK-16868)

Author: huangzhaowei <carlmartinmax@gmail.com>

Closes #14530 from SaintBacchus/SPARK-16868.
parent 1c9a386c
No related branches found
No related tags found
No related merge requests found
......@@ -77,6 +77,10 @@ class StorageStatusListener(conf: SparkConf) extends SparkListener {
val maxMem = blockManagerAdded.maxMem
val storageStatus = new StorageStatus(blockManagerId, maxMem)
executorIdToStorageStatus(executorId) = storageStatus
// Try to remove the dead storage status if same executor register the block manager twice.
deadExecutorStorageStatus.zipWithIndex.find(_._1.blockManagerId.executorId == executorId)
.foreach(toRemoveExecutor => deadExecutorStorageStatus.remove(toRemoveExecutor._2))
}
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment