Skip to content
Snippets Groups Projects
Commit 4e4831b8 authored by Zhen Peng's avatar Zhen Peng Committed by Aaron Davidson
Browse files

[SPARK-1886] check executor id existence when executor exit

Author: Zhen Peng <zhenpeng01@baidu.com>

Closes #827 from zhpengg/bugfix-executor-id-not-found and squashes the following commits:

cd8bb65 [Zhen Peng] bugfix: check executor id existence when executor exit
parent 75a03277
No related branches found
No related tags found
No related merge requests found
...@@ -263,14 +263,20 @@ private[spark] class Worker( ...@@ -263,14 +263,20 @@ private[spark] class Worker(
} }
val fullId = appId + "/" + execId val fullId = appId + "/" + execId
if (ExecutorState.isFinished(state)) { if (ExecutorState.isFinished(state)) {
val executor = executors(fullId) executors.get(fullId) match {
logInfo("Executor " + fullId + " finished with state " + state + case Some(executor) =>
message.map(" message " + _).getOrElse("") + logInfo("Executor " + fullId + " finished with state " + state +
exitStatus.map(" exitStatus " + _).getOrElse("")) message.map(" message " + _).getOrElse("") +
executors -= fullId exitStatus.map(" exitStatus " + _).getOrElse(""))
finishedExecutors(fullId) = executor executors -= fullId
coresUsed -= executor.cores finishedExecutors(fullId) = executor
memoryUsed -= executor.memory coresUsed -= executor.cores
memoryUsed -= executor.memory
case None =>
logInfo("Unknown Executor " + fullId + " finished with state " + state +
message.map(" message " + _).getOrElse("") +
exitStatus.map(" exitStatus " + _).getOrElse(""))
}
} }
case KillExecutor(masterUrl, appId, execId) => case KillExecutor(masterUrl, appId, execId) =>
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment