-
- Downloads
[SPARK-4266] [Web-UI] Reduce stage page load time.
The commit changes the java script used to show/hide additional metrics in order to reduce page load time. SPARK-4016 significantly increased page load time for the stage page when stages had a lot (thousands or tens of thousands) of tasks, due to the additional Javascript to hide some metrics by default and stripe the tables. This commit reduces page load time in two ways: (1) Now, all of the metrics that are hidden by default are hidden by setting "display: none;" using CSS for the page, rather than hiding them using javascript after the page loads. Without this change, for stages with thousands of tasks, there was a few second delay after page load, where first the additional metrics were shown, and then after a delay were hidden once the relevant JS finished running. (2) CSS is used to stripe all of the tables except for the summary table. The summary table needs javascript to do the striping because some rows are hidden, but the javascript striping is slower, which again resulted in a delay when it was used for the task table (where for a few seconds after page load, all of the rows in the task table would be white, while the browser finished running the JS to stripe the table). cc pwendell This change is intended to be backported to 1.2 to avoid a regression in UI performance when users run large jobs. Author: Kay Ousterhout <kayousterhout@gmail.com> Closes #3328 from kayousterhout/SPARK-4266 and squashes the following commits: f964091 [Kay Ousterhout] [SPARK-4266] [Web-UI] Reduce stage page load time.
Showing
- core/src/main/resources/org/apache/spark/ui/static/additional-metrics.js 2 additions, 7 deletions...esources/org/apache/spark/ui/static/additional-metrics.js
- core/src/main/resources/org/apache/spark/ui/static/table.js 13 additions, 11 deletionscore/src/main/resources/org/apache/spark/ui/static/table.js
- core/src/main/resources/org/apache/spark/ui/static/webui.css 6 additions, 0 deletionscore/src/main/resources/org/apache/spark/ui/static/webui.css
- core/src/main/scala/org/apache/spark/ui/UIUtils.scala 5 additions, 6 deletionscore/src/main/scala/org/apache/spark/ui/UIUtils.scala
- core/src/main/scala/org/apache/spark/ui/exec/ExecutorsPage.scala 1 addition, 1 deletion...c/main/scala/org/apache/spark/ui/exec/ExecutorsPage.scala
- core/src/main/scala/org/apache/spark/ui/jobs/ExecutorTable.scala 1 addition, 1 deletion...c/main/scala/org/apache/spark/ui/jobs/ExecutorTable.scala
- core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala 8 additions, 1 deletioncore/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
- core/src/main/scala/org/apache/spark/ui/jobs/TaskDetailsClassNames.scala 3 additions, 0 deletions...cala/org/apache/spark/ui/jobs/TaskDetailsClassNames.scala
Loading
Please register or sign in to comment