Skip to content
Snippets Groups Projects
Commit e8813be6 authored by WangTaoTheTonic's avatar WangTaoTheTonic Committed by Thomas Graves
Browse files

[SPARK-4095][YARN][Minor]extract val isLaunchingDriver in ClientBase

Instead of checking if `args.userClass` is null repeatedly, we extract it to an global val as in `ApplicationMaster`.

Author: WangTaoTheTonic <barneystinson@aliyun.com>

Closes #2954 from WangTaoTheTonic/MemUnit and squashes the following commits:

13bda20 [WangTaoTheTonic] extract val isLaunchingDriver in ClientBase
parent 47346cd0
No related branches found
No related tags found
No related merge requests found
......@@ -55,6 +55,7 @@ private[spark] trait ClientBase extends Logging {
protected val amMemoryOverhead = args.amMemoryOverhead // MB
protected val executorMemoryOverhead = args.executorMemoryOverhead // MB
private val distCacheMgr = new ClientDistributedCacheManager()
private val isLaunchingDriver = args.userClass != null
/**
* Fail fast if we have requested more resources per container than is available in the cluster.
......@@ -267,7 +268,6 @@ private[spark] trait ClientBase extends Logging {
// Note that to warn the user about the deprecation in cluster mode, some code from
// SparkConf#validateSettings() is duplicated here (to avoid triggering the condition
// described above).
val isLaunchingDriver = args.userClass != null
if (isLaunchingDriver) {
sys.env.get("SPARK_JAVA_OPTS").foreach { value =>
val warning =
......@@ -344,7 +344,6 @@ private[spark] trait ClientBase extends Logging {
}
// Include driver-specific java options if we are launching a driver
val isLaunchingDriver = args.userClass != null
if (isLaunchingDriver) {
sparkConf.getOption("spark.driver.extraJavaOptions")
.orElse(sys.env.get("SPARK_JAVA_OPTS"))
......@@ -357,7 +356,7 @@ private[spark] trait ClientBase extends Logging {
javaOpts += ("-Dspark.yarn.app.container.log.dir=" + ApplicationConstants.LOG_DIR_EXPANSION_VAR)
val userClass =
if (args.userClass != null) {
if (isLaunchingDriver) {
Seq("--class", YarnSparkHadoopUtil.escapeForShell(args.userClass))
} else {
Nil
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment