Skip to content
Snippets Groups Projects
Commit 630bd5fd authored by zsxwing's avatar zsxwing Committed by Kousuke Saruta
Browse files

[SPARK-8702] [WEBUI] Avoid massive concating strings in Javascript

When there are massive tasks, such as `sc.parallelize(1 to 100000, 10000).count()`, the generated JS codes have a lot of string concatenations in the stage page, nearly 40 string concatenations for one task.

We can generate the whole string for a task instead of execution string concatenations in the browser.

Before this patch, the load time of the page is about 21 seconds.
![screen shot 2015-06-29 at 6 44 04 pm](https://cloud.githubusercontent.com/assets/1000778/8406644/eb55ed18-1e90-11e5-9ad5-50d27ad1dff1.png)

After this patch, it reduces to about 17 seconds.

![screen shot 2015-06-29 at 6 47 34 pm](https://cloud.githubusercontent.com/assets/1000778/8406665/087003ca-1e91-11e5-80a8-3485aa9adafa.png)

One disadvantage is that the generated JS codes become hard to read.

Author: zsxwing <zsxwing@gmail.com>

Closes #7082 from zsxwing/js-string and squashes the following commits:

b29231d [zsxwing] Avoid massive concating strings in Javascript
parent 660c6cec
No related branches found
No related tags found
No related merge requests found
...@@ -572,55 +572,55 @@ private[ui] class StagePage(parent: StagesTab) extends WebUIPage("stage") { ...@@ -572,55 +572,55 @@ private[ui] class StagePage(parent: StagesTab) extends WebUIPage("stage") {
val attempt = taskInfo.attempt val attempt = taskInfo.attempt
val timelineObject = val timelineObject =
s""" s"""
{ |{
'className': 'task task-assignment-timeline-object', |'className': 'task task-assignment-timeline-object',
'group': '$executorId', |'group': '$executorId',
'content': '<div class="task-assignment-timeline-content"' + |'content': '<div class="task-assignment-timeline-content"
'data-toggle="tooltip" data-placement="top"' + |data-toggle="tooltip" data-placement="top"
'data-html="true" data-container="body"' + |data-html="true" data-container="body"
'data-title="${s"Task " + index + " (attempt " + attempt + ")"}<br>' + |data-title="${s"Task " + index + " (attempt " + attempt + ")"}<br>
'Status: ${taskInfo.status}<br>' + |Status: ${taskInfo.status}<br>
'Launch Time: ${UIUtils.formatDate(new Date(launchTime))}' + |Launch Time: ${UIUtils.formatDate(new Date(launchTime))}
'${ |${
if (!taskInfo.running) { if (!taskInfo.running) {
s"""<br>Finish Time: ${UIUtils.formatDate(new Date(finishTime))}""" s"""<br>Finish Time: ${UIUtils.formatDate(new Date(finishTime))}"""
} else { } else {
"" ""
} }
}' + }
'<br>Scheduler Delay: $schedulerDelay ms' + |<br>Scheduler Delay: $schedulerDelay ms
'<br>Task Deserialization Time: ${UIUtils.formatDuration(deserializationTime)}' + |<br>Task Deserialization Time: ${UIUtils.formatDuration(deserializationTime)}
'<br>Shuffle Read Time: ${UIUtils.formatDuration(shuffleReadTime)}' + |<br>Shuffle Read Time: ${UIUtils.formatDuration(shuffleReadTime)}
'<br>Executor Computing Time: ${UIUtils.formatDuration(executorComputingTime)}' + |<br>Executor Computing Time: ${UIUtils.formatDuration(executorComputingTime)}
'<br>Shuffle Write Time: ${UIUtils.formatDuration(shuffleWriteTime)}' + |<br>Shuffle Write Time: ${UIUtils.formatDuration(shuffleWriteTime)}
'<br>Result Serialization Time: ${UIUtils.formatDuration(serializationTime)}' + |<br>Result Serialization Time: ${UIUtils.formatDuration(serializationTime)}
'<br>Getting Result Time: ${UIUtils.formatDuration(gettingResultTime)}">' + |<br>Getting Result Time: ${UIUtils.formatDuration(gettingResultTime)}">
'<svg class="task-assignment-timeline-duration-bar">' + |<svg class="task-assignment-timeline-duration-bar">
'<rect class="scheduler-delay-proportion" ' + |<rect class="scheduler-delay-proportion"
'x="$schedulerDelayProportionPos%" y="0px" height="26px"' + |x="$schedulerDelayProportionPos%" y="0px" height="26px"
'width="$schedulerDelayProportion%""></rect>' + |width="$schedulerDelayProportion%""></rect>
'<rect class="deserialization-time-proportion" '+ |<rect class="deserialization-time-proportion"
'x="$deserializationTimeProportionPos%" y="0px" height="26px"' + |x="$deserializationTimeProportionPos%" y="0px" height="26px"
'width="$deserializationTimeProportion%"></rect>' + |width="$deserializationTimeProportion%"></rect>
'<rect class="shuffle-read-time-proportion" ' + |<rect class="shuffle-read-time-proportion"
'x="$shuffleReadTimeProportionPos%" y="0px" height="26px"' + |x="$shuffleReadTimeProportionPos%" y="0px" height="26px"
'width="$shuffleReadTimeProportion%"></rect>' + |width="$shuffleReadTimeProportion%"></rect>
'<rect class="executor-runtime-proportion" ' + |<rect class="executor-runtime-proportion"
'x="$executorRuntimeProportionPos%" y="0px" height="26px"' + |x="$executorRuntimeProportionPos%" y="0px" height="26px"
'width="$executorComputingTimeProportion%"></rect>' + |width="$executorComputingTimeProportion%"></rect>
'<rect class="shuffle-write-time-proportion" ' + |<rect class="shuffle-write-time-proportion"
'x="$shuffleWriteTimeProportionPos%" y="0px" height="26px"' + |x="$shuffleWriteTimeProportionPos%" y="0px" height="26px"
'width="$shuffleWriteTimeProportion%"></rect>' + |width="$shuffleWriteTimeProportion%"></rect>
'<rect class="serialization-time-proportion" ' + |<rect class="serialization-time-proportion"
'x="$serializationTimeProportionPos%" y="0px" height="26px"' + |x="$serializationTimeProportionPos%" y="0px" height="26px"
'width="$serializationTimeProportion%"></rect>' + |width="$serializationTimeProportion%"></rect>
'<rect class="getting-result-time-proportion" ' + |<rect class="getting-result-time-proportion"
'x="$gettingResultTimeProportionPos%" y="0px" height="26px"' + |x="$gettingResultTimeProportionPos%" y="0px" height="26px"
'width="$gettingResultTimeProportion%"></rect></svg>', |width="$gettingResultTimeProportion%"></rect></svg>',
'start': new Date($launchTime), |'start': new Date($launchTime),
'end': new Date($finishTime) |'end': new Date($finishTime)
} |}
""" |""".stripMargin.replaceAll("\n", " ")
timelineObject timelineObject
}.mkString("[", ",", "]") }.mkString("[", ",", "]")
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment