Skip to content
Snippets Groups Projects
Commit 5bcb9a7f authored by Weiqing Yang's avatar Weiqing Yang Committed by Marcelo Vanzin
Browse files

[SPARK-18417][YARN] Define 'spark.yarn.am.port' in yarn config object

## What changes were proposed in this pull request?
This PR is to define 'spark.yarn.am.port' in yarn config.scala just like other Yarn configurations. That makes code easier to maintain.

## How was this patch tested?
Build passed & tested some Yarn unit tests.

Author: Weiqing Yang <yangweiqing001@gmail.com>

Closes #15858 from weiqingy/yarn.
parent 2afdaa98
No related branches found
No related tags found
No related merge requests found
......@@ -429,7 +429,7 @@ private[spark] class ApplicationMaster(
}
private def runExecutorLauncher(securityMgr: SecurityManager): Unit = {
val port = sparkConf.getInt("spark.yarn.am.port", 0)
val port = sparkConf.get(AM_PORT)
rpcEnv = RpcEnv.create("sparkYarnAM", Utils.localHostName, port, sparkConf, securityMgr,
clientMode = true)
val driverRef = waitForSparkDriver()
......
......@@ -40,6 +40,11 @@ package object config {
.timeConf(TimeUnit.MILLISECONDS)
.createOptional
private[spark] val AM_PORT =
ConfigBuilder("spark.yarn.am.port")
.intConf
.createWithDefault(0)
private[spark] val EXECUTOR_ATTEMPT_FAILURE_VALIDITY_INTERVAL_MS =
ConfigBuilder("spark.yarn.executor.failuresValidityInterval")
.doc("Interval after which Executor failures will be considered independent and not " +
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment