-
- Downloads
[SPARK-6735][YARN] Add window based executor failure tracking mechanism for long running service
This work is based on twinkle-sachdeva 's proposal. In parallel to such mechanism for AM failures, here add similar mechanism for executor failure tracking, this is useful for long running Spark service to mitigate the executor failure problems. Please help to review, tgravescs sryza and vanzin Author: jerryshao <sshao@hortonworks.com> Closes #10241 from jerryshao/SPARK-6735.
Showing
- docs/running-on-yarn.md 8 additions, 0 deletionsdocs/running-on-yarn.md
- yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala 2 additions, 2 deletions.../src/main/scala/org/apache/spark/deploy/yarn/Client.scala
- yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala 29 additions, 5 deletions...in/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
- yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala 8 additions, 1 deletion.../src/main/scala/org/apache/spark/deploy/yarn/config.scala
- yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala 46 additions, 0 deletions...ala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala
Loading
Please register or sign in to comment