-
- Downloads
[SPARK-15487][WEB UI] Spark Master UI to reverse proxy Application and Workers UI
## What changes were proposed in this pull request? This pull request adds the functionality to enable accessing worker and application UI through master UI itself. Thus helps in accessing SparkUI when running spark cluster in closed networks e.g. Kubernetes. Cluster admin needs to expose only spark master UI and rest of the UIs can be in the private network, master UI will reverse proxy the connection request to corresponding resource. It adds the path for workers/application UIs as WorkerUI: <http/https>://master-publicIP:<port>/target/workerID/ ApplicationUI: <http/https>://master-publicIP:<port>/target/appID/ This makes it easy for users to easily protect the Spark master cluster access by putting some reverse proxy e.g. https://github.com/bitly/oauth2_proxy ## How was this patch tested? The functionality has been tested manually and there is a unit test too for testing access to worker UI with reverse proxy address. pwendell bomeng BryanCutler can you please review it, thanks. Author: Gurvinder Singh <gurvinder.singh@uninett.no> Closes #13950 from gurvindersingh/rproxy.
Showing
- core/pom.xml 11 additions, 1 deletioncore/pom.xml
- core/src/main/scala/org/apache/spark/SparkContext.scala 3 additions, 0 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/deploy/master/Master.scala 18 additions, 0 deletions...rc/main/scala/org/apache/spark/deploy/master/Master.scala
- core/src/main/scala/org/apache/spark/deploy/master/ui/ApplicationPage.scala 9 additions, 4 deletions...a/org/apache/spark/deploy/master/ui/ApplicationPage.scala
- core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala 9 additions, 3 deletions.../scala/org/apache/spark/deploy/master/ui/MasterPage.scala
- core/src/main/scala/org/apache/spark/deploy/master/ui/MasterWebUI.scala 16 additions, 0 deletions...scala/org/apache/spark/deploy/master/ui/MasterWebUI.scala
- core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala 5 additions, 1 deletion...scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
- core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala 3 additions, 0 deletions...rc/main/scala/org/apache/spark/deploy/worker/Worker.scala
- core/src/main/scala/org/apache/spark/ui/JettyUtils.scala 85 additions, 0 deletionscore/src/main/scala/org/apache/spark/ui/JettyUtils.scala
- core/src/main/scala/org/apache/spark/ui/UIUtils.scala 12 additions, 0 deletionscore/src/main/scala/org/apache/spark/ui/UIUtils.scala
- core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala 27 additions, 0 deletions...st/scala/org/apache/spark/deploy/master/MasterSuite.scala
- core/src/test/scala/org/apache/spark/ui/UISuite.scala 37 additions, 0 deletionscore/src/test/scala/org/apache/spark/ui/UISuite.scala
- docs/configuration.md 14 additions, 0 deletionsdocs/configuration.md
- pom.xml 14 additions, 0 deletionspom.xml
- repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkILoopInit.scala 12 additions, 1 deletion...src/main/scala/org/apache/spark/repl/SparkILoopInit.scala
- repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoop.scala 12 additions, 1 deletion....11/src/main/scala/org/apache/spark/repl/SparkILoop.scala
Loading
Please register or sign in to comment