Skip to content
Snippets Groups Projects
Commit a21a3bbe authored by Sandeep Singh's avatar Sandeep Singh Committed by Wenchen Fan
Browse files

[SPARK-15087][MINOR][DOC] Follow Up: Fix the Comments

## What changes were proposed in this pull request?
Remove the Comment, since it not longer applies. see the discussion here(https://github.com/apache/spark/pull/12865#discussion-diff-61946906)

Author: Sandeep Singh <sandeep@techaddict.me>

Closes #12953 from techaddict/SPARK-15087-FOLLOW-UP.
parent cc95f1ed
No related branches found
No related tags found
No related merge requests found
...@@ -389,11 +389,6 @@ private[spark] class TaskSchedulerImpl( ...@@ -389,11 +389,6 @@ private[spark] class TaskSchedulerImpl(
// (taskId, stageId, stageAttemptId, accumUpdates) // (taskId, stageId, stageAttemptId, accumUpdates)
val accumUpdatesWithTaskIds: Array[(Long, Int, Int, Seq[AccumulableInfo])] = synchronized { val accumUpdatesWithTaskIds: Array[(Long, Int, Int, Seq[AccumulableInfo])] = synchronized {
accumUpdates.flatMap { case (id, updates) => accumUpdates.flatMap { case (id, updates) =>
// We should call `acc.value` here as we are at driver side now. However, the RPC framework
// optimizes local message delivery so that messages do not need to de serialized and
// deserialized. This brings trouble to the accumulator framework, which depends on
// serialization to set the `atDriverSide` flag. Here we call `acc.localValue` instead to
// be more robust about this issue.
val accInfos = updates.map(acc => acc.toInfo(Some(acc.value), None)) val accInfos = updates.map(acc => acc.toInfo(Some(acc.value), None))
taskIdToTaskSetManager.get(id).map { taskSetMgr => taskIdToTaskSetManager.get(id).map { taskSetMgr =>
(id, taskSetMgr.stageId, taskSetMgr.taskSet.stageAttemptId, accInfos) (id, taskSetMgr.stageId, taskSetMgr.taskSet.stageAttemptId, accInfos)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment