-
- Downloads
[SPARK-13407] Guard against garbage-collected accumulators in TaskMetrics.fromAccumulatorUpdates
`TaskMetrics.fromAccumulatorUpdates()` can fail if accumulators have been garbage-collected on the driver. To guard against this, this patch introduces `ListenerTaskMetrics`, a subclass of `TaskMetrics` which is used only in `TaskMetrics.fromAccumulatorUpdates()` and which eliminates the need to access the original accumulators on the driver. Author: Josh Rosen <joshrosen@databricks.com> Closes #11276 from JoshRosen/accum-updates-fix.
Showing
- core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala 29 additions, 26 deletions...rc/main/scala/org/apache/spark/executor/TaskMetrics.scala
- core/src/test/scala/org/apache/spark/executor/TaskMetricsSuite.scala 4 additions, 6 deletions...st/scala/org/apache/spark/executor/TaskMetricsSuite.scala
Please register or sign in to comment