-
- Downloads
[SPARK-5124][Core] A standard RPC interface and an Akka implementation
This PR added a standard internal RPC interface for Spark and an Akka implementation. See [the design document](https://issues.apache.org/jira/secure/attachment/12698710/Pluggable%20RPC%20-%20draft%202.pdf) for more details. I will split the whole work into multiple PRs to make it easier for code review. This is the first PR and avoid to touch too many files. Author: zsxwing <zsxwing@gmail.com> Closes #4588 from zsxwing/rpc-part1 and squashes the following commits: fe3df4c [zsxwing] Move registerEndpoint and use actorSystem.dispatcher in asyncSetupEndpointRefByURI f6f3287 [zsxwing] Remove RpcEndpointRef.toURI 8bd1097 [zsxwing] Fix docs and the code style f459380 [zsxwing] Add RpcAddress.fromURI and rename urls to uris b221398 [zsxwing] Move send methods above ask methods 15cfd7b [zsxwing] Merge branch 'master' into rpc-part1 9ffa997 [zsxwing] Fix MiMa tests 78a1733 [zsxwing] Merge remote-tracking branch 'origin/master' into rpc-part1 385b9c3 [zsxwing] Fix the code style and add docs 2cc3f78 [zsxwing] Add an asynchronous version of setupEndpointRefByUrl e8dfec3 [zsxwing] Remove 'sendWithReply(message: Any, sender: RpcEndpointRef): Unit' 08564ae [zsxwing] Add RpcEnvFactory to create RpcEnv e5df4ca [zsxwing] Handle AkkaFailure(e) in Actor ec7c5b0 [zsxwing] Fix docs 7fc95e1 [zsxwing] Implement askWithReply in RpcEndpointRef 9288406 [zsxwing] Document thread-safety for setupThreadSafeEndpoint 3007c09 [zsxwing] Move setupDriverEndpointRef to RpcUtils and rename to makeDriverRef c425022 [zsxwing] Fix the code style 5f87700 [zsxwing] Move the logical of processing message to a private function 3e56123 [zsxwing] Use lazy to eliminate CountDownLatch 07f128f [zsxwing] Remove ActionScheduler.scala 4d34191 [zsxwing] Remove scheduler from RpcEnv 7cdd95e [zsxwing] Add docs for RpcEnv 51e6667 [zsxwing] Add 'sender' to RpcCallContext and rename the parameter of receiveAndReply to 'context' ffc1280 [zsxwing] Rename 'fail' to 'sendFailure' and other minor code style changes 28e6d0f [zsxwing] Add onXXX for network events and remove the companion objects of network events 3751c97 [zsxwing] Rename RpcResponse to RpcCallContext fe7d1ff [zsxwing] Add explicit reply in rpc 7b9e0c9 [zsxwing] Fix the indentation 04a106e [zsxwing] Remove NopCancellable and add a const NOP in object SettableCancellable 2a579f4 [zsxwing] Remove RpcEnv.systemName 155b987 [zsxwing] Change newURI to uriOf and add some comments 45b2317 [zsxwing] A standard RPC interface and An Akka implementation
Showing
- core/src/main/scala/org/apache/spark/SparkEnv.scala 28 additions, 14 deletionscore/src/main/scala/org/apache/spark/SparkEnv.scala
- core/src/main/scala/org/apache/spark/deploy/worker/DriverWrapper.scala 5 additions, 6 deletions.../scala/org/apache/spark/deploy/worker/DriverWrapper.scala
- core/src/main/scala/org/apache/spark/deploy/worker/WorkerWatcher.scala 32 additions, 27 deletions.../scala/org/apache/spark/deploy/worker/WorkerWatcher.scala
- core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala 1 addition, 1 deletion.../apache/spark/executor/CoarseGrainedExecutorBackend.scala
- core/src/main/scala/org/apache/spark/rpc/RpcEnv.scala 429 additions, 0 deletionscore/src/main/scala/org/apache/spark/rpc/RpcEnv.scala
- core/src/main/scala/org/apache/spark/rpc/akka/AkkaRpcEnv.scala 318 additions, 0 deletions...src/main/scala/org/apache/spark/rpc/akka/AkkaRpcEnv.scala
- core/src/main/scala/org/apache/spark/scheduler/OutputCommitCoordinator.scala 17 additions, 20 deletions.../org/apache/spark/scheduler/OutputCommitCoordinator.scala
- core/src/main/scala/org/apache/spark/util/AkkaUtils.scala 1 addition, 1 deletioncore/src/main/scala/org/apache/spark/util/AkkaUtils.scala
- core/src/main/scala/org/apache/spark/util/RpcUtils.scala 35 additions, 0 deletionscore/src/main/scala/org/apache/spark/util/RpcUtils.scala
- core/src/test/scala/org/apache/spark/deploy/worker/WorkerWatcherSuite.scala 22 additions, 16 deletions...a/org/apache/spark/deploy/worker/WorkerWatcherSuite.scala
- core/src/test/scala/org/apache/spark/rpc/RpcEnvSuite.scala 525 additions, 0 deletionscore/src/test/scala/org/apache/spark/rpc/RpcEnvSuite.scala
- core/src/test/scala/org/apache/spark/rpc/akka/AkkaRpcEnvSuite.scala 50 additions, 0 deletions...est/scala/org/apache/spark/rpc/akka/AkkaRpcEnvSuite.scala
- project/MimaExcludes.scala 3 additions, 1 deletionproject/MimaExcludes.scala
Loading
Please register or sign in to comment