-
- Downloads
[SPARK-9817][YARN] Improve the locality calculation of containers by taking...
[SPARK-9817][YARN] Improve the locality calculation of containers by taking pending container requests into consideraion This is a follow-up PR to further improve the locality calculation by considering the pending container's request. Since the locality preferences of tasks may be shifted from time to time, current localities of pending container requests may not fully match the new preferences, this PR improve it by removing outdated, unmatched container requests and replace with new requests. sryza please help to review, thanks a lot. Author: jerryshao <sshao@hortonworks.com> Closes #8100 from jerryshao/SPARK-9817.
Showing
- yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala 1 addition, 1 deletion...cala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
- yarn/src/main/scala/org/apache/spark/deploy/yarn/LocalityPreferredContainerPlacementStrategy.scala 52 additions, 8 deletions...oy/yarn/LocalityPreferredContainerPlacementStrategy.scala
- yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala 60 additions, 13 deletions...in/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
- yarn/src/test/scala/org/apache/spark/deploy/yarn/ContainerPlacementStrategySuite.scala 33 additions, 5 deletions...e/spark/deploy/yarn/ContainerPlacementStrategySuite.scala
- yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala 13 additions, 13 deletions...ala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala
Loading
Please register or sign in to comment