-
- Downloads
[SPARK-21694][MESOS] Support Mesos CNI network labels
JIRA ticket: https://issues.apache.org/jira/browse/SPARK-21694 ## What changes were proposed in this pull request? Spark already supports launching containers attached to a given CNI network by specifying it via the config `spark.mesos.network.name`. This PR adds support to pass in network labels to CNI plugins via a new config option `spark.mesos.network.labels`. These network labels are key-value pairs that are set in the `NetworkInfo` of both the driver and executor tasks. More details in the related Mesos documentation: http://mesos.apache.org/documentation/latest/cni/#mesos-meta-data-to-cni-plugins ## How was this patch tested? Unit tests, for both driver and executor tasks. Manual integration test to submit a job with the `spark.mesos.network.labels` option, hit the mesos/state.json endpoint, and check that the labels are set in the driver and executor tasks. ArtRand skonto Author: Susan X. Huynh <xhuynh@mesosphere.com> Closes #18910 from susanxhuynh/sh-mesos-cni-labels.
Showing
- docs/running-on-mesos.md 14 additions, 0 deletionsdocs/running-on-mesos.md
- resource-managers/mesos/src/main/scala/org/apache/spark/deploy/mesos/config.scala 17 additions, 2 deletions...src/main/scala/org/apache/spark/deploy/mesos/config.scala
- resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala 1 addition, 1 deletion...er/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala
- resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala 7 additions, 2 deletions...k/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala
- resource-managers/mesos/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterSchedulerSuite.scala 7 additions, 2 deletions.../scheduler/cluster/mesos/MesosClusterSchedulerSuite.scala
- resource-managers/mesos/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackendSuite.scala 7 additions, 2 deletions...uster/mesos/MesosCoarseGrainedSchedulerBackendSuite.scala
Loading
Please register or sign in to comment