-
- Downloads
[SPARK-1774] Respect SparkSubmit --jars on YARN (client)
SparkSubmit ignores `--jars` for YARN client. This is a bug. This PR also automatically adds the application jar to `spark.jar`. Previously, when running as yarn-client, you must specify the jar additionally through `--files` (because `--jars` didn't work). Now you don't have to explicitly specify it through either. Tested on a YARN cluster. Author: Andrew Or <andrewor14@gmail.com> Closes #710 from andrewor14/yarn-jars and squashes the following commits: 35d1928 [Andrew Or] Merge branch 'master' of github.com:apache/spark into yarn-jars c27bf6c [Andrew Or] For yarn-cluster and python, do not add primaryResource to spark.jar c92c5bf [Andrew Or] Minor cleanups 269f9f3 [Andrew Or] Fix format 013d840 [Andrew Or] Fix tests 1407474 [Andrew Or] Merge branch 'master' of github.com:apache/spark into yarn-jars 3bb75e8 [Andrew Or] Allow SparkSubmit --jars to take effect in yarn-client mode
Showing
- core/src/main/scala/org/apache/spark/SparkContext.scala 1 addition, 1 deletioncore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala 23 additions, 16 deletions.../src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
- core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala 77 additions, 33 deletions...test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala
- yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala 1 addition, 3 deletions.../scala/org/apache/spark/deploy/yarn/ClientArguments.scala
Loading
Please register or sign in to comment