Skip to content
Snippets Groups Projects
Commit 4e3f4b93 authored by zsxwing's avatar zsxwing Committed by Andrew Or
Browse files

[SPARK-9829] [WEBUI] Display the update value for peak execution memory

The peak execution memory is not correct because it shows the sum of finished tasks' values when a task finishes.

This PR fixes it by using the update value rather than the accumulator value.

Author: zsxwing <zsxwing@gmail.com>

Closes #8121 from zsxwing/SPARK-9829.
parent a807fcbe
No related branches found
No related tags found
No related merge requests found
...@@ -860,7 +860,7 @@ private[ui] class TaskDataSource( ...@@ -860,7 +860,7 @@ private[ui] class TaskDataSource(
} }
val peakExecutionMemoryUsed = taskInternalAccumulables val peakExecutionMemoryUsed = taskInternalAccumulables
.find { acc => acc.name == InternalAccumulator.PEAK_EXECUTION_MEMORY } .find { acc => acc.name == InternalAccumulator.PEAK_EXECUTION_MEMORY }
.map { acc => acc.value.toLong } .map { acc => acc.update.getOrElse("0").toLong }
.getOrElse(0L) .getOrElse(0L)
val maybeInput = metrics.flatMap(_.inputMetrics) val maybeInput = metrics.flatMap(_.inputMetrics)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment