-
- Downloads
Merge pull request #127 from kayousterhout/consolidate_schedulers
Deduplicate Local and Cluster schedulers. The code in LocalScheduler/LocalTaskSetManager was nearly identical to the code in ClusterScheduler/ClusterTaskSetManager. The redundancy made making updating the schedulers unnecessarily painful and error- prone. This commit combines the two into a single TaskScheduler/ TaskSetManager. Unfortunately the diff makes this change look much more invasive than it is -- TaskScheduler.scala is only superficially changed (names updated, overrides removed) from the old ClusterScheduler.scala, and the same with TaskSetManager.scala. Thanks @rxin for suggesting this change!
No related branches found
No related tags found
Showing
- core/src/main/scala/org/apache/spark/SparkContext.scala 25 additions, 16 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/scheduler/ExecutorLossReason.scala 1 addition, 1 deletion...scala/org/apache/spark/scheduler/ExecutorLossReason.scala
- core/src/main/scala/org/apache/spark/scheduler/SchedulerBackend.scala 2 additions, 2 deletions...n/scala/org/apache/spark/scheduler/SchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskResultGetter.scala 4 additions, 5 deletions...n/scala/org/apache/spark/scheduler/TaskResultGetter.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala 6 additions, 5 deletions...main/scala/org/apache/spark/scheduler/TaskScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala 21 additions, 20 deletions.../scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala 689 additions, 15 deletions...ain/scala/org/apache/spark/scheduler/TaskSetManager.scala
- core/src/main/scala/org/apache/spark/scheduler/WorkerOffer.scala 1 addition, 1 deletion...c/main/scala/org/apache/spark/scheduler/WorkerOffer.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManager.scala 0 additions, 713 deletions...pache/spark/scheduler/cluster/ClusterTaskSetManager.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala 4 additions, 2 deletions...ark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/SimrSchedulerBackend.scala 3 additions, 1 deletion...apache/spark/scheduler/cluster/SimrSchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala 4 additions, 2 deletions...spark/scheduler/cluster/SparkDeploySchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala 3 additions, 2 deletions...scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala 3 additions, 4 deletions...spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/local/LocalBackend.scala 108 additions, 0 deletions...scala/org/apache/spark/scheduler/local/LocalBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/local/LocalScheduler.scala 0 additions, 222 deletions...ala/org/apache/spark/scheduler/local/LocalScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/local/LocalTaskSetManager.scala 0 additions, 191 deletions...rg/apache/spark/scheduler/local/LocalTaskSetManager.scala
- core/src/test/scala/org/apache/spark/FailureSuite.scala 2 additions, 2 deletionscore/src/test/scala/org/apache/spark/FailureSuite.scala
- core/src/test/scala/org/apache/spark/SparkContextSchedulerCreationSuite.scala 25 additions, 33 deletions...org/apache/spark/SparkContextSchedulerCreationSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/ClusterSchedulerSuite.scala 7 additions, 9 deletions...la/org/apache/spark/scheduler/ClusterSchedulerSuite.scala
Loading
Please register or sign in to comment