-
- Downloads
Merge pull request #23 from jerryshao/multi-user
Add Spark multi-user support for standalone mode and Mesos This PR add multi-user support for Spark both standalone mode and Mesos (coarse and fine grained ) mode, user can specify the user name who submit app through environment variable `SPARK_USER` or use default one. Executor will communicate with Hadoop using specified user name. Also I fixed one bug in JobLogger when different user wrote job log to specified folder which has no right file permission. I separate previous [PR750](https://github.com/mesos/spark/pull/750) into two PRs, in this PR I only solve multi-user support problem. I will try to solve security auth problem in subsequent PR because security auth is a complicated problem especially for Shark Server like long-run app (both Kerberos TGT and HDFS delegation token should be renewed or re-created through app's run time).
No related branches found
No related tags found
Showing
- core/src/main/scala/org/apache/spark/SparkContext.scala 10 additions, 0 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala 15 additions, 3 deletions.../main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
- core/src/main/scala/org/apache/spark/executor/Executor.scala 5 additions, 2 deletionscore/src/main/scala/org/apache/spark/executor/Executor.scala
- core/src/main/scala/org/apache/spark/scheduler/JobLogger.scala 384 additions, 373 deletions...src/main/scala/org/apache/spark/scheduler/JobLogger.scala
- core/src/test/scala/org/apache/spark/scheduler/JobLoggerSuite.scala 3 additions, 1 deletion...est/scala/org/apache/spark/scheduler/JobLoggerSuite.scala
Loading
Please register or sign in to comment