-
- Downloads
[SPARK-10790] [YARN] Fix initial executor number not set issue and consolidate the codes
This bug is introduced in [SPARK-9092](https://issues.apache.org/jira/browse/SPARK-9092), `targetExecutorNumber` should use `minExecutors` if `initialExecutors` is not set. Using 0 instead will meet the problem as mentioned in [SPARK-10790](https://issues.apache.org/jira/browse/SPARK-10790). Also consolidate and simplify some similar code snippets to keep the consistent semantics. Author: jerryshao <sshao@hortonworks.com> Closes #8910 from jerryshao/SPARK-10790.
Showing
- yarn/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala 1 addition, 19 deletions.../scala/org/apache/spark/deploy/yarn/ClientArguments.scala
- yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala 1 addition, 5 deletions...in/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
- yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala 23 additions, 0 deletions...la/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala
- yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterSchedulerBackend.scala 2 additions, 16 deletions...spark/scheduler/cluster/YarnClusterSchedulerBackend.scala
Loading
Please register or sign in to comment