-
- Downloads
SPARK-1319: Fix scheduler to account for tasks using > 1 CPUs.
Move CPUS_PER_TASK to TaskSchedulerImpl as the value is a constant and use it in both Mesos and CoarseGrained scheduler backends. Thanks @kayousterhout for the design discussion Author: Shivaram Venkataraman <shivaram@eecs.berkeley.edu> Closes #219 from shivaram/multi-cpus and squashes the following commits: 5c7d685 [Shivaram Venkataraman] Don't pass availableCpus to TaskSetManager 260e4d5 [Shivaram Venkataraman] Add a check for non-zero CPUs in TaskSetManager 73fcf6f [Shivaram Venkataraman] Add documentation for spark.task.cpus 647bc45 [Shivaram Venkataraman] Fix scheduler to account for tasks using > 1 CPUs. Move CPUS_PER_TASK to TaskSchedulerImpl as the value is a constant and use it in both Mesos and CoarseGrained scheduler backends.
Showing
- core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala 15 additions, 10 deletions.../scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala 1 addition, 5 deletions...ain/scala/org/apache/spark/scheduler/TaskSetManager.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala 2 additions, 2 deletions...ark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala 1 addition, 1 deletion...spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala
- core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala 40 additions, 2 deletions...a/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala 40 additions, 43 deletions...cala/org/apache/spark/scheduler/TaskSetManagerSuite.scala
- docs/configuration.md 7 additions, 0 deletionsdocs/configuration.md
Loading
Please register or sign in to comment