-
- Downloads
[SPARK-23475][WEBUI] Skipped stages should be evicted before completed stages
## What changes were proposed in this pull request? The root cause of missing completed stages is because `cleanupStages` will never remove skipped stages. This PR changes the logic to always remove skipped stage first. This is safe since the job itself contains enough information to render skipped stages in the UI. ## How was this patch tested? The new unit tests. Author: Shixiong Zhu <zsxwing@gmail.com> Closes #20656 from zsxwing/SPARK-23475.
Showing
- core/src/main/scala/org/apache/spark/status/AppStatusListener.scala 4 additions, 1 deletion...ain/scala/org/apache/spark/status/AppStatusListener.scala
- core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala 36 additions, 0 deletions...cala/org/apache/spark/status/AppStatusListenerSuite.scala
Loading
Please register or sign in to comment