Skip to content
Snippets Groups Projects
Commit 074f5023 authored by Patrick Wendell's avatar Patrick Wendell
Browse files

Merge pull request #396 from pwendell/executor-env

Setting load defaults to true in executor

This preserves the behavior in earlier releases. If properties are set for the executors via `spark-env.sh` on the slaves, then they should take precedence over spark defaults. This is useful for if system administrators are setting properties for a standalone cluster, such as shuffle locations.

/cc @andrewor14 who initially reported this issue.
parents 82e2b92c cfb1e6c1
No related branches found
No related tags found
No related merge requests found
......@@ -57,7 +57,7 @@ private[spark] class Executor(
Utils.setCustomHostname(slaveHostname)
// Set spark.* properties from executor arg
val conf = new SparkConf(false)
val conf = new SparkConf(true)
conf.setAll(properties)
// If we are in yarn mode, systems can have different disk layouts so we must set it
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment