-
- Downloads
[SPARK-17062][MESOS] add conf option to mesos dispatcher
Adds --conf option to set spark configuration properties in mesos dispacther. Properties provided with --conf take precedence over properties within the properties file. The reason for this PR is that for simple configuration or testing purposes we need to provide a property file (ideally a shared one for a cluster) even if we just provide a single property. Manually tested. Author: Stavros Kontopoulos <st.kontopoulos@gmail.com> Author: Stavros Kontopoulos <stavros.kontopoulos@lightbend.com> Closes #14650 from skonto/dipatcher_conf.
Showing
- core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala 4 additions, 14 deletions.../src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
- core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala 2 additions, 4 deletions.../scala/org/apache/spark/deploy/SparkSubmitArguments.scala
- core/src/main/scala/org/apache/spark/util/CommandLineUtils.scala 56 additions, 0 deletions...c/main/scala/org/apache/spark/util/CommandLineUtils.scala
- core/src/main/scala/org/apache/spark/util/Utils.scala 14 additions, 0 deletionscore/src/main/scala/org/apache/spark/util/Utils.scala
- core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala 26 additions, 17 deletions...test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala
- mesos/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcher.scala 6 additions, 3 deletions...rg/apache/spark/deploy/mesos/MesosClusterDispatcher.scala
- mesos/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcherArguments.scala 55 additions, 15 deletions.../spark/deploy/mesos/MesosClusterDispatcherArguments.scala
- mesos/src/test/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcherArgumentsSuite.scala 63 additions, 0 deletions...k/deploy/mesos/MesosClusterDispatcherArgumentsSuite.scala
- mesos/src/test/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcherSuite.scala 40 additions, 0 deletions...ache/spark/deploy/mesos/MesosClusterDispatcherSuite.scala
Loading
Please register or sign in to comment