-
- Downloads
[SPARK-6602][Core] Replace direct use of Akka with Spark RPC interface - part 1
This PR replaced the following `Actor`s to `RpcEndpoint`: 1. HeartbeatReceiver 1. ExecutorActor 1. BlockManagerMasterActor 1. BlockManagerSlaveActor 1. CoarseGrainedExecutorBackend and subclasses 1. CoarseGrainedSchedulerBackend.DriverActor This is the first PR. I will split the work of SPARK-6602 to several PRs for code review. Author: zsxwing <zsxwing@gmail.com> Closes #5268 from zsxwing/rpc-rewrite and squashes the following commits: 287e9f8 [zsxwing] Fix the code style 26c56b7 [zsxwing] Merge branch 'master' into rpc-rewrite 9cc825a [zsxwing] Rmove setupThreadSafeEndpoint and add ThreadSafeRpcEndpoint 30a9036 [zsxwing] Make self return null after stopping RpcEndpointRef; fix docs and error messages 705245d [zsxwing] Fix some bugs after rebasing the changes on the master 003cf80 [zsxwing] Update CoarseGrainedExecutorBackend and CoarseGrainedSchedulerBackend to use RpcEndpoint 7d0e6dc [zsxwing] Update BlockManagerSlaveActor to use RpcEndpoint f5d6543 [zsxwing] Update BlockManagerMaster to use RpcEndpoint 30e3f9f [zsxwing] Update ExecutorActor to use RpcEndpoint 478b443 [zsxwing] Update HeartbeatReceiver to use RpcEndpoint
Showing
- core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala 43 additions, 23 deletionscore/src/main/scala/org/apache/spark/HeartbeatReceiver.scala
- core/src/main/scala/org/apache/spark/SparkContext.scala 12 additions, 11 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/SparkEnv.scala 8 additions, 5 deletionscore/src/main/scala/org/apache/spark/SparkEnv.scala
- core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala 43 additions, 36 deletions.../apache/spark/executor/CoarseGrainedExecutorBackend.scala
- core/src/main/scala/org/apache/spark/executor/Executor.scala 6 additions, 12 deletionscore/src/main/scala/org/apache/spark/executor/Executor.scala
- core/src/main/scala/org/apache/spark/executor/ExecutorEndpoint.scala 10 additions, 8 deletions...in/scala/org/apache/spark/executor/ExecutorEndpoint.scala
- core/src/main/scala/org/apache/spark/rpc/RpcEnv.scala 18 additions, 21 deletionscore/src/main/scala/org/apache/spark/rpc/RpcEnv.scala
- core/src/main/scala/org/apache/spark/rpc/akka/AkkaRpcEnv.scala 1 addition, 9 deletions...src/main/scala/org/apache/spark/rpc/akka/AkkaRpcEnv.scala
- core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala 2 additions, 9 deletions.../main/scala/org/apache/spark/scheduler/DAGScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedClusterMessage.scala 5 additions, 1 deletion...spark/scheduler/cluster/CoarseGrainedClusterMessage.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala 76 additions, 72 deletions...ark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/ExecutorData.scala 4 additions, 4 deletions...ala/org/apache/spark/scheduler/cluster/ExecutorData.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/SimrSchedulerBackend.scala 5 additions, 8 deletions...apache/spark/scheduler/cluster/SimrSchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala 6 additions, 8 deletions...spark/scheduler/cluster/SparkDeploySchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala 47 additions, 46 deletions...apache/spark/scheduler/cluster/YarnSchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala 2 additions, 2 deletions...scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
- core/src/main/scala/org/apache/spark/scheduler/local/LocalBackend.scala 27 additions, 21 deletions...scala/org/apache/spark/scheduler/local/LocalBackend.scala
- core/src/main/scala/org/apache/spark/storage/BlockManager.scala 11 additions, 11 deletions...rc/main/scala/org/apache/spark/storage/BlockManager.scala
- core/src/main/scala/org/apache/spark/storage/BlockManagerMaster.scala 32 additions, 40 deletions...n/scala/org/apache/spark/storage/BlockManagerMaster.scala
- core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala 58 additions, 61 deletions...org/apache/spark/storage/BlockManagerMasterEndpoint.scala
Loading
Please register or sign in to comment