-
- Downloads
[SPARK-5529][CORE]Add expireDeadHosts in HeartbeatReceiver
If a blockManager has not send heartBeat more than 120s, BlockManagerMasterActor will remove it. But coarseGrainedSchedulerBackend can only remove executor after an DisassociatedEvent. We should expireDeadHosts at HeartbeatReceiver. Author: Hong Shen <hongshen@tencent.com> Closes #4363 from shenh062326/my_change3 and squashes the following commits: 2c9a46a [Hong Shen] Change some code style. 1a042ff [Hong Shen] Change some code style. 2dc456e [Hong Shen] Change some code style. d221493 [Hong Shen] Fix test failed 7448ac6 [Hong Shen] A minor change in sparkContext and heartbeatReceiver b904aed [Hong Shen] Fix failed test 52725af [Hong Shen] Remove assert in SparkContext.killExecutors 5bedcb8 [Hong Shen] Remove assert in SparkContext.killExecutors a858fb5 [Hong Shen] A minor change in HeartbeatReceiver 3e221d9 [Hong Shen] A minor change in HeartbeatReceiver 6bab7aa [Hong Shen] Change a code style. 07952f3 [Hong Shen] Change configs name and code style. ce9257e [Hong Shen] Fix test failed bccd515 [Hong Shen] Fix test failed 8e77408 [Hong Shen] Fix test failed c1dfda1 [Hong Shen] Fix test failed e197e20 [Hong Shen] Fix test failed fb5df97 [Hong Shen] Remove ExpireDeadHosts in BlockManagerMessages b5c0441 [Hong Shen] Remove expireDeadHosts in BlockManagerMasterActor c922cb0 [Hong Shen] Add expireDeadHosts in HeartbeatReceiver
Showing
- core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala 59 additions, 6 deletionscore/src/main/scala/org/apache/spark/HeartbeatReceiver.scala
- core/src/main/scala/org/apache/spark/SparkContext.scala 11 additions, 4 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala 5 additions, 1 deletion...main/scala/org/apache/spark/scheduler/TaskScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala 1 addition, 1 deletion.../scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
- core/src/main/scala/org/apache/spark/storage/BlockManagerMasterActor.scala 1 addition, 35 deletions...la/org/apache/spark/storage/BlockManagerMasterActor.scala
- core/src/main/scala/org/apache/spark/storage/BlockManagerMessages.scala 0 additions, 2 deletions...scala/org/apache/spark/storage/BlockManagerMessages.scala
- core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala 2 additions, 0 deletions.../scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala
Loading
Please register or sign in to comment