Skip to content
Snippets Groups Projects
Unverified Commit 2ab8551e authored by Yuming Wang's avatar Yuming Wang Committed by Sean Owen
Browse files

[SPARK-18645][DEPLOY] Fix spark-daemon.sh arguments error lead to throws Unrecognized option

## What changes were proposed in this pull request?

spark-daemon.sh will lost single quotes around after #15338. as follows:
```
execute_command nice -n 0 bash /opt/cloudera/parcels/SPARK-2.1.0-cdh5.4.3.d20161129-21.04.38/lib/spark/bin/spark-submit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --name Thrift JDBC/ODBC Server --conf spark.driver.extraJavaOptions=-XX:+UseG1GC -XX:-HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp
```
With this fix, as follows:
```
execute_command nice -n 0 bash /opt/cloudera/parcels/SPARK-2.1.0-cdh5.4.3.d20161129-21.04.38/lib/spark/bin/spark-submit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --name 'Thrift JDBC/ODBC Server' --conf 'spark.driver.extraJavaOptions=-XX:+UseG1GC -XX:-HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp'
```

## How was this patch tested?

- Manual tests
- Build the package and start-thriftserver.sh with `--conf 'spark.driver.extraJavaOptions=-XX:+UseG1GC -XX:-HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp'`

Author: Yuming Wang <wgyumg@gmail.com>

Closes #16079 from wangyum/SPARK-18645.
parent dbf842b7
No related branches found
No related tags found
No related merge requests found
......@@ -124,9 +124,8 @@ if [ "$SPARK_NICENESS" = "" ]; then
fi
execute_command() {
local command="$@"
if [ -z ${SPARK_NO_DAEMONIZE+set} ]; then
nohup -- $command >> $log 2>&1 < /dev/null &
nohup -- "$@" >> $log 2>&1 < /dev/null &
newpid="$!"
echo "$newpid" > "$pid"
......@@ -143,12 +142,12 @@ execute_command() {
sleep 2
# Check if the process has died; in that case we'll tail the log so the user can see
if [[ ! $(ps -p "$newpid" -o comm=) =~ "java" ]]; then
echo "failed to launch $command:"
echo "failed to launch: $@"
tail -2 "$log" | sed 's/^/ /'
echo "full log in $log"
fi
else
$command
"$@"
fi
}
......@@ -176,11 +175,11 @@ run_command() {
case "$mode" in
(class)
execute_command nice -n "$SPARK_NICENESS" "${SPARK_HOME}"/bin/spark-class $command $@
execute_command nice -n "$SPARK_NICENESS" "${SPARK_HOME}"/bin/spark-class "$command" "$@"
;;
(submit)
execute_command nice -n "$SPARK_NICENESS" bash "${SPARK_HOME}"/bin/spark-submit --class $command $@
execute_command nice -n "$SPARK_NICENESS" bash "${SPARK_HOME}"/bin/spark-submit --class "$command" "$@"
;;
(*)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment