-
- Downloads
[SPARK-16106][CORE] TaskSchedulerImpl should properly track executors added to existing hosts
## What changes were proposed in this pull request? TaskSchedulerImpl used to only set `newExecAvailable` when a new *host* was added, not when a new executor was added to an existing host. It also didn't update some internal state tracking live executors until a task was scheduled on the executor. This patch changes it to properly update as soon as it knows about a new executor. ## How was this patch tested? added a unit test, ran everything via jenkins. Author: Imran Rashid <irashid@cloudera.com> Closes #13826 from squito/SPARK-16106_executorByHosts.
Showing
- core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala 5 additions, 3 deletions.../scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
- core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala 106 additions, 62 deletions...a/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala
Loading
Please register or sign in to comment