-
- Downloads
Merge pull request #14 from kayousterhout/untangle_scheduler
Improved organization of scheduling packages. This commit does not change any code -- only file organization. Please let me know if there was some masterminded strategy behind the existing organization that I failed to understand! There are two components of this change: (1) Moving files out of the cluster package, and down a level to the scheduling package. These files are all used by the local scheduler in addition to the cluster scheduler(s), so should not be in the cluster package. As a result of this change, none of the files in the local package reference files in the cluster package. (2) Moving the mesos package to within the cluster package. The mesos scheduling code is for a cluster, and represents a specific case of cluster scheduling (the Mesos-related classes often subclass cluster scheduling classes). Thus, the most logical place for it seems to be within the cluster package. The one thing about the scheduling code that seems a little funny to me is the naming of the SchedulerBackends. The StandaloneSchedulerBackend is not just for Standalone mode, but instead is used by Mesos coarse grained mode and Yarn, and the backend that *is* just for Standalone mode is instead called SparkDeploySchedulerBackend. I didn't change this because I wasn't sure if there was a reason for this naming that I'm just not aware of.
No related branches found
No related tags found
Showing
- core/src/main/scala/org/apache/spark/SparkContext.scala 2 additions, 2 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala 0 additions, 1 deletion.../main/scala/org/apache/spark/scheduler/DAGScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/DAGSchedulerEvent.scala 0 additions, 1 deletion.../scala/org/apache/spark/scheduler/DAGSchedulerEvent.scala
- core/src/main/scala/org/apache/spark/scheduler/JobLogger.scala 0 additions, 1 deletion...src/main/scala/org/apache/spark/scheduler/JobLogger.scala
- core/src/main/scala/org/apache/spark/scheduler/Pool.scala 2 additions, 2 deletionscore/src/main/scala/org/apache/spark/scheduler/Pool.scala
- core/src/main/scala/org/apache/spark/scheduler/Schedulable.scala 2 additions, 2 deletions...c/main/scala/org/apache/spark/scheduler/Schedulable.scala
- core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala 1 addition, 1 deletion...scala/org/apache/spark/scheduler/SchedulableBuilder.scala
- core/src/main/scala/org/apache/spark/scheduler/SchedulingAlgorithm.scala 1 addition, 1 deletion...cala/org/apache/spark/scheduler/SchedulingAlgorithm.scala
- core/src/main/scala/org/apache/spark/scheduler/SchedulingMode.scala 1 addition, 1 deletion...ain/scala/org/apache/spark/scheduler/SchedulingMode.scala
- core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala 0 additions, 1 deletion...main/scala/org/apache/spark/scheduler/SparkListener.scala
- core/src/main/scala/org/apache/spark/scheduler/StageInfo.scala 1 addition, 1 deletion...src/main/scala/org/apache/spark/scheduler/StageInfo.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala 1 addition, 1 deletion...in/scala/org/apache/spark/scheduler/TaskDescription.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala 1 addition, 1 deletion.../src/main/scala/org/apache/spark/scheduler/TaskInfo.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskLocality.scala 1 addition, 1 deletion.../main/scala/org/apache/spark/scheduler/TaskLocality.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala 3 additions, 2 deletions...main/scala/org/apache/spark/scheduler/TaskScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerListener.scala 0 additions, 1 deletion...la/org/apache/spark/scheduler/TaskSchedulerListener.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala 1 addition, 2 deletions...ain/scala/org/apache/spark/scheduler/TaskSetManager.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterScheduler.scala 1 addition, 1 deletion...org/apache/spark/scheduler/cluster/ClusterScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManager.scala 3 additions, 6 deletions...pache/spark/scheduler/cluster/ClusterTaskSetManager.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/StandaloneClusterMessage.scala 1 addition, 0 deletions...he/spark/scheduler/cluster/StandaloneClusterMessage.scala
Loading
Please register or sign in to comment