-
- Downloads
[SPARK-9731] Standalone scheduling incorrect cores if spark.executor.cores is not set
The issue only happens if `spark.executor.cores` is not set and executor memory is set to a high value. For example, if we have a worker with 4G and 10 cores and we set `spark.executor.memory` to 3G, then only 1 core is assigned to the executor. The correct number should be 10 cores. I've added a unit test to illustrate the issue. Author: Carson Wang <carson.wang@intel.com> Closes #8017 from carsonwang/SPARK-9731 and squashes the following commits: d09ec48 [Carson Wang] Fix code style 86b651f [Carson Wang] Simplify the code 943cc4c [Carson Wang] fix scheduling correct cores to executors
Showing
- core/src/main/scala/org/apache/spark/deploy/master/Master.scala 14 additions, 12 deletions...rc/main/scala/org/apache/spark/deploy/master/Master.scala
- core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala 15 additions, 0 deletions...st/scala/org/apache/spark/deploy/master/MasterSuite.scala
Loading
Please register or sign in to comment