-
- Downloads
[SPARK-1112, 2156] Bootstrap to fetch the driver's Spark properties.
This is an alternative solution to #1124 . Before launching the executor backend, we first fetch driver's spark properties and use it to overwrite executor's spark properties. This should be better than #1124. @pwendell Are there spark properties that might be different on the driver and on the executors? Author: Xiangrui Meng <meng@databricks.com> Closes #1132 from mengxr/akka-bootstrap and squashes the following commits: 77ff32d [Xiangrui Meng] organize imports 68e1dfb [Xiangrui Meng] use timeout from AkkaUtils; remove props from RegisteredExecutor 46d332d [Xiangrui Meng] fix a test 7947c18 [Xiangrui Meng] increase slack size for akka 4ab696a [Xiangrui Meng] bootstrap to retrieve driver spark conf
Showing
- core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala 37 additions, 29 deletions.../apache/spark/executor/CoarseGrainedExecutorBackend.scala
- core/src/main/scala/org/apache/spark/executor/Executor.scala 1 addition, 1 deletioncore/src/main/scala/org/apache/spark/executor/Executor.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedClusterMessage.scala 3 additions, 3 deletions...spark/scheduler/cluster/CoarseGrainedClusterMessage.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala 9 additions, 6 deletions...ark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
- core/src/main/scala/org/apache/spark/util/AkkaUtils.scala 3 additions, 0 deletionscore/src/main/scala/org/apache/spark/util/AkkaUtils.scala
- core/src/test/scala/org/apache/spark/scheduler/CoarseGrainedSchedulerBackendSuite.scala 1 addition, 1 deletion.../spark/scheduler/CoarseGrainedSchedulerBackendSuite.scala
Loading
Please register or sign in to comment