-
- Downloads
[SPARK-18537][WEB UI] Add a REST api to serve spark streaming information
## What changes were proposed in this pull request? This PR is an inheritance from #16000, and is a completion of #15904. **Description** - Augment the `org.apache.spark.status.api.v1` package for serving streaming information. - Retrieve the streaming information through StreamingJobProgressListener. > this api should cover exceptly the same amount of information as you can get from the web interface > the implementation is base on the current REST implementation of spark-core > and will be available for running applications only > > https://issues.apache.org/jira/browse/SPARK-18537 ## How was this patch tested? Local test. Author: saturday_s <shi.indetail@gmail.com> Author: Chan Chor Pang <ChorPang.Chan@access-company.com> Author: peterCPChan <universknight@gmail.com> Closes #16253 from saturday-shi/SPARK-18537.
Showing
- core/src/main/scala/org/apache/spark/ui/SparkUI.scala 8 additions, 0 deletionscore/src/main/scala/org/apache/spark/ui/SparkUI.scala
- project/MimaExcludes.scala 5 additions, 1 deletionproject/MimaExcludes.scala
- streaming/src/main/java/org/apache/spark/status/api/v1/streaming/BatchStatus.java 30 additions, 0 deletions...org/apache/spark/status/api/v1/streaming/BatchStatus.java
- streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/AllBatchesResource.scala 78 additions, 0 deletions...he/spark/status/api/v1/streaming/AllBatchesResource.scala
- streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/AllOutputOperationsResource.scala 66 additions, 0 deletions...status/api/v1/streaming/AllOutputOperationsResource.scala
- streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/AllReceiversResource.scala 76 additions, 0 deletions.../spark/status/api/v1/streaming/AllReceiversResource.scala
- streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/ApiStreamingApp.scala 42 additions, 0 deletions...pache/spark/status/api/v1/streaming/ApiStreamingApp.scala
- streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/ApiStreamingRootResource.scala 74 additions, 0 deletions...rk/status/api/v1/streaming/ApiStreamingRootResource.scala
- streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/OneBatchResource.scala 35 additions, 0 deletions...ache/spark/status/api/v1/streaming/OneBatchResource.scala
- streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/OneOutputOperationResource.scala 39 additions, 0 deletions.../status/api/v1/streaming/OneOutputOperationResource.scala
- streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/OneReceiverResource.scala 35 additions, 0 deletions...e/spark/status/api/v1/streaming/OneReceiverResource.scala
- streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/StreamingStatisticsResource.scala 64 additions, 0 deletions...status/api/v1/streaming/StreamingStatisticsResource.scala
- streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/api.scala 75 additions, 0 deletions.../scala/org/apache/spark/status/api/v1/streaming/api.scala
- streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala 4 additions, 1 deletion...n/scala/org/apache/spark/streaming/StreamingContext.scala
- streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaStreamingListener.scala 14 additions, 0 deletions...ache/spark/streaming/api/java/JavaStreamingListener.scala
- streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaStreamingListenerWrapper.scala 5 additions, 0 deletions...ark/streaming/api/java/JavaStreamingListenerWrapper.scala
- streaming/src/main/scala/org/apache/spark/streaming/scheduler/StreamingListener.scala 6 additions, 0 deletions.../apache/spark/streaming/scheduler/StreamingListener.scala
- streaming/src/main/scala/org/apache/spark/streaming/scheduler/StreamingListenerBus.scala 2 additions, 0 deletions...ache/spark/streaming/scheduler/StreamingListenerBus.scala
- streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingJobProgressListener.scala 10 additions, 2 deletions...che/spark/streaming/ui/StreamingJobProgressListener.scala
- streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala 2 additions, 1 deletion...n/scala/org/apache/spark/streaming/ui/StreamingPage.scala
Loading
Please register or sign in to comment