-
- Downloads
[SPARK-3468] [WEBUI] Timeline-View feature
I sometimes trouble-shoot and analyse the cause of long time spending job. At the time, I find the stages which spends long time or fails, then I find the tasks which spends long time or fails, next I analyse the proportion of each phase in a task. Another case, I find executors which spends long time for running a task and analyse the details of a task. In such situation, I think it's helpful to visualize timeline for each application, job, task and the details of proportion of activity for each task. I added 3 timeline features to existing Web UI. [Application Timeline View] This view shows following things. * When each executor was added/removed and the reason why it's removed. * When each job was started/finished. * Status of each job.  [Stage Timeline View] Similar to Application Timeline View, this view shows following things. * When each executor was added/removed and the reason why it's removed. * When each job was started/finished. * Status of each stage.  [Task Assignment Timeline View] This view shows following things. * When each task started/finished * How long each task spent and the proportion. * Status of each task. * Where each task ran on.  All the view above is zoomable by mouse wheel action and scrollable by drag action. Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp> Closes #2342 from sarutak/timeline-viewer-feature and squashes the following commits: 11fe67d [Kousuke Saruta] Fixed conflict 79ac03d [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature a91abd3 [Kousuke Saruta] Merge branch 'master' of https://github.com/apache/spark into timeline-viewer-feature ef34a5b [Kousuke Saruta] Implement tooltip using bootstrap b09d0c5 [Kousuke Saruta] Move `stroke` and `fill` attribute of rect elements to css d3c63c8 [Kousuke Saruta] Fixed a little bit bugs a36291b [Kousuke Saruta] Merge branch 'master' of https://github.com/apache/spark into timeline-viewer-feature 28714b6 [Kousuke Saruta] Fixed highlight issue 0dc4278 [Kousuke Saruta] Addressed most of Patrics's feedbacks 8110acf [Kousuke Saruta] Added scroll limit to Job timeline 974a64a [Kousuke Saruta] Removed unused function ee7a7f0 [Kousuke Saruta] Refactored 6a91872 [Kousuke Saruta] Temporary commit 6693f34 [Kousuke Saruta] Added link to job/stage box in the timeline in order to move to corresponding row when we click 8f88222 [Kousuke Saruta] Added job/stage description aeed4b1 [Kousuke Saruta] Removed stage timeline fc1696c [Kousuke Saruta] Merge branch 'timeline-viewer-feature' of github.com:sarutak/spark into timeline-viewer-feature 999ccd4 [Kousuke Saruta] Improved scalability 0fc6a31 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature 19815ae [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature 68b7540 [Kousuke Saruta] Merge branch 'timeline-viewer-feature' of github.com:sarutak/spark into timeline-viewer-feature 52b5f0b [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature dec85db [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature fcdab7d [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature dab7cc1 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature 09cce97 [Kousuke Saruta] Cleanuped 16f82cf [Kousuke Saruta] Cleanuped 9fb522e [Kousuke Saruta] Cleanuped d05f2c2 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature e85e9aa [Kousuke Saruta] Cleanup: Added TimelineViewUtils.scala a76e569 [Kousuke Saruta] Removed unused setting in timeline-view.css 5ce1b21 [Kousuke Saruta] Added vis.min.js, vis.min.css and vis.map to .rat-exclude 082f709 [Kousuke Saruta] Added Timeline-View feature for Applications, Jobs and Stages
Showing
- .rat-excludes 3 additions, 0 deletions.rat-excludes
- core/src/main/resources/org/apache/spark/ui/static/timeline-view.css 182 additions, 0 deletions...in/resources/org/apache/spark/ui/static/timeline-view.css
- core/src/main/resources/org/apache/spark/ui/static/timeline-view.js 168 additions, 0 deletions...ain/resources/org/apache/spark/ui/static/timeline-view.js
- core/src/main/resources/org/apache/spark/ui/static/vis.map 1 addition, 0 deletionscore/src/main/resources/org/apache/spark/ui/static/vis.map
- core/src/main/resources/org/apache/spark/ui/static/vis.min.css 1 addition, 0 deletions...src/main/resources/org/apache/spark/ui/static/vis.min.css
- core/src/main/resources/org/apache/spark/ui/static/vis.min.js 39 additions, 0 deletions.../src/main/resources/org/apache/spark/ui/static/vis.min.js
- core/src/main/scala/org/apache/spark/scheduler/StageInfo.scala 12 additions, 0 deletions...src/main/scala/org/apache/spark/scheduler/StageInfo.scala
- core/src/main/scala/org/apache/spark/ui/UIUtils.scala 5 additions, 0 deletionscore/src/main/scala/org/apache/spark/ui/UIUtils.scala
- core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala 11 additions, 0 deletions...rc/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala
- core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala 182 additions, 18 deletions...src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala
- core/src/main/scala/org/apache/spark/ui/jobs/JobPage.scala 163 additions, 11 deletionscore/src/main/scala/org/apache/spark/ui/jobs/JobPage.scala
- core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala 7 additions, 0 deletions.../scala/org/apache/spark/ui/jobs/JobProgressListener.scala
- core/src/main/scala/org/apache/spark/ui/jobs/JobsTab.scala 4 additions, 2 deletionscore/src/main/scala/org/apache/spark/ui/jobs/JobsTab.scala
- core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala 2 additions, 1 deletion.../src/main/scala/org/apache/spark/ui/jobs/StageTable.scala
- core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala 5 additions, 0 deletionscore/src/main/scala/org/apache/spark/ui/jobs/UIData.scala
Loading
Please register or sign in to comment