Skip to content
Snippets Groups Projects
Commit a4a081d1 authored by Timothy Chen's avatar Timothy Chen Committed by Reynold Xin
Browse files

[SPARK-13414][MESOS] Allow multiple dispatchers to be launched.

## What changes were proposed in this pull request?

Users might want to start multiple mesos dispatchers, as each dispatcher can potentially be part of different roles and used for multi-tenancy.

To allow multiple Mesos dispatchers to be launched, we need to be able to specify a instance number when starting the dispatcher daemon.

## How was the this patch tested?

Manual testing

Author: Timothy Chen <tnachen@gmail.com>

Closes #11281 from tnachen/multiple_cluster_dispatchers.
parent 6ce7c481
No related branches found
No related tags found
No related merge requests found
...@@ -37,5 +37,8 @@ if [ "$SPARK_MESOS_DISPATCHER_HOST" = "" ]; then ...@@ -37,5 +37,8 @@ if [ "$SPARK_MESOS_DISPATCHER_HOST" = "" ]; then
SPARK_MESOS_DISPATCHER_HOST=`hostname` SPARK_MESOS_DISPATCHER_HOST=`hostname`
fi fi
if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
SPARK_MESOS_DISPATCHER_NUM=1
fi
"${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher 1 --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT "$@" "${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher $SPARK_MESOS_DISPATCHER_NUM --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT "$@"
...@@ -24,5 +24,10 @@ fi ...@@ -24,5 +24,10 @@ fi
. "${SPARK_HOME}/sbin/spark-config.sh" . "${SPARK_HOME}/sbin/spark-config.sh"
"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher 1 if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
SPARK_MESOS_DISPATCHER_NUM=1
fi
"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher \
$SPARK_MESOS_DISPATCHER_NUM
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment