-
- Downloads
[SPARK-19567][CORE][SCHEDULER] Support some Schedulable variables immutability and access
## What changes were proposed in this pull request? Some `Schedulable` Entities(`Pool` and `TaskSetManager`) variables need refactoring for _immutability_ and _access modifiers_ levels as follows: - From `var` to `val` (if there is no requirement): This is important to support immutability as much as possible. - Sample => `Pool`: `weight`, `minShare`, `priority`, `name` and `taskSetSchedulingAlgorithm`. - Access modifiers: Specially, `var`s access needs to be restricted from other parts of codebase to prevent potential side effects. - `TaskSetManager`: `tasksSuccessful`, `totalResultSize`, `calculatedTasks` etc... This PR is related with #15604 and has been created seperatedly to keep patch content as isolated and to help the reviewers. ## How was this patch tested? Added new UTs and existing UT coverage. Author: erenavsarogullari <erenavsarogullari@gmail.com> Closes #16905 from erenavsarogullari/SPARK-19567.
Showing
- core/src/main/scala/org/apache/spark/scheduler/Pool.scala 6 additions, 6 deletionscore/src/main/scala/org/apache/spark/scheduler/Pool.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala 12 additions, 7 deletions.../scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala 19 additions, 17 deletions...ain/scala/org/apache/spark/scheduler/TaskSetManager.scala
- core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala 4 additions, 4 deletions.../scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala 2 additions, 2 deletions.../apache/spark/scheduler/ExternalClusterManagerSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/PoolSuite.scala 6 additions, 0 deletions...src/test/scala/org/apache/spark/scheduler/PoolSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala 9 additions, 3 deletions...a/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala
Loading
Please register or sign in to comment