-
- Downloads
[SPARK-20191][YARN] Crate wrapper for RackResolver so tests can override it.
Current test code tries to override the RackResolver used by setting configuration params, but because YARN libs statically initialize the resolver the first time it's used, that means that those configs don't really take effect during Spark tests. This change adds a wrapper class that easily allows tests to override the behavior of the resolver for the Spark code that uses it. Author: Marcelo Vanzin <vanzin@cloudera.com> Closes #17508 from vanzin/SPARK-20191. (cherry picked from commit 0736980f) Signed-off-by:Marcelo Vanzin <vanzin@cloudera.com>
Showing
- yarn/src/main/scala/org/apache/spark/deploy/yarn/LocalityPreferredContainerPlacementStrategy.scala 3 additions, 3 deletions...oy/yarn/LocalityPreferredContainerPlacementStrategy.scala
- yarn/src/main/scala/org/apache/spark/deploy/yarn/SparkRackResolver.scala 40 additions, 0 deletions...cala/org/apache/spark/deploy/yarn/SparkRackResolver.scala
- yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala 4 additions, 9 deletions...in/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
- yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClient.scala 1 addition, 1 deletion...ain/scala/org/apache/spark/deploy/yarn/YarnRMClient.scala
- yarn/src/test/scala/org/apache/spark/deploy/yarn/LocalityPlacementStrategySuite.scala 2 additions, 6 deletions...he/spark/deploy/yarn/LocalityPlacementStrategySuite.scala
- yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala 6 additions, 16 deletions...ala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala
Loading
Please register or sign in to comment