Skip to content
Snippets Groups Projects
Commit 983fa2d6 authored by Josh Rosen's avatar Josh Rosen
Browse files

[SPARK-13407] Guard against garbage-collected accumulators in TaskMetrics.fromAccumulatorUpdates 

`TaskMetrics.fromAccumulatorUpdates()` can fail if accumulators have been garbage-collected on the driver. To guard against this, this patch introduces `ListenerTaskMetrics`, a subclass of `TaskMetrics` which is used only in `TaskMetrics.fromAccumulatorUpdates()` and which eliminates the need to access the original accumulators on the driver.

Author: Josh Rosen <joshrosen@databricks.com>

Closes #11276 from JoshRosen/accum-updates-fix.
parent 091f6a78
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment