-
- Downloads
[SPARK-3353] parent stage should have lower stage id.
Previously parent stages had higher stage id, but parent stages are executed first. This pull request changes the behavior so parent stages would have lower stage id. For example, command: ```scala sc.parallelize(1 to 10).map(x=>(x,x)).reduceByKey(_+_).count ``` breaks down into 2 stages. The old web UI:  Web UI with this patch:  Author: Reynold Xin <rxin@apache.org> Closes #2273 from rxin/lower-stage-id and squashes the following commits: abbb4c6 [Reynold Xin] Fixed SparkListenerSuite. 0e02379 [Reynold Xin] Updated DAGSchedulerSuite. 54ccea3 [Reynold Xin] [SPARK-3353] parent stage should have lower stage id.
Showing
- core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala 2 additions, 2 deletions.../main/scala/org/apache/spark/scheduler/DAGScheduler.scala
- core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala 16 additions, 9 deletions.../scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/SparkListenerSuite.scala 1 addition, 1 deletion...scala/org/apache/spark/scheduler/SparkListenerSuite.scala
Loading
Please register or sign in to comment