-
- Downloads
[SPARK-14062][YARN] Fix log4j and upload metrics.properties automatically with distributed cache
## What changes were proposed in this pull request? 1. Currently log4j which uses distributed cache only adds to AM's classpath, not executor's, this is introduced in #9118, which breaks the original meaning of that PR, so here add log4j file to the classpath of both AM and executors. 2. Automatically upload metrics.properties to distributed cache, so that it could be used by remote driver and executors implicitly. ## How was this patch tested? Unit test and integration test is done. Author: jerryshao <sshao@hortonworks.com> Closes #11885 from jerryshao/SPARK-14062.
Showing
- yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala 27 additions, 44 deletions.../src/main/scala/org/apache/spark/deploy/yarn/Client.scala
- yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala 1 addition, 2 deletions...scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala
- yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala 3 additions, 4 deletions...test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala
Loading
Please register or sign in to comment