-
- Downloads
[SPARK-18836][CORE] Serialize one copy of task metrics in DAGScheduler
## What changes were proposed in this pull request? Right now we serialize the empty task metrics once per task – Since this is shared across all tasks we could use the same serialized task metrics across all tasks of a stage. ## How was this patch tested? - [x] Run tests on EC2 to measure performance improvement Author: Shivaram Venkataraman <shivaram@cs.berkeley.edu> Closes #16261 from shivaram/task-metrics-one-copy.
Showing
- core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala 3 additions, 2 deletions.../main/scala/org/apache/spark/scheduler/DAGScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala 5 additions, 4 deletions...rc/main/scala/org/apache/spark/scheduler/ResultTask.scala
- core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala 6 additions, 5 deletions...ain/scala/org/apache/spark/scheduler/ShuffleMapTask.scala
- core/src/main/scala/org/apache/spark/scheduler/Task.scala 8 additions, 2 deletionscore/src/main/scala/org/apache/spark/scheduler/Task.scala
- core/src/test/scala/org/apache/spark/executor/ExecutorSuite.scala 3 additions, 1 deletion.../test/scala/org/apache/spark/executor/ExecutorSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/FakeTask.scala 9 additions, 1 deletion.../src/test/scala/org/apache/spark/scheduler/FakeTask.scala
- core/src/test/scala/org/apache/spark/scheduler/TaskContextSuite.scala 4 additions, 2 deletions...t/scala/org/apache/spark/scheduler/TaskContextSuite.scala
Loading
Please register or sign in to comment