-
- Downloads
[SPARK-13465] Add a task failure listener to TaskContext
## What changes were proposed in this pull request? TaskContext supports task completion callback, which gets called regardless of task failures. However, there is no way for the listener to know if there is an error. This patch adds a new listener that gets called when a task fails. ## How was the this patch tested? New unit test case and integration test case covering the code path Author: Reynold Xin <rxin@databricks.com> Closes #11340 from rxin/SPARK-13465.
Showing
- core/src/main/scala/org/apache/spark/TaskContext.scala 26 additions, 2 deletionscore/src/main/scala/org/apache/spark/TaskContext.scala
- core/src/main/scala/org/apache/spark/TaskContextImpl.scala 26 additions, 7 deletionscore/src/main/scala/org/apache/spark/TaskContextImpl.scala
- core/src/main/scala/org/apache/spark/scheduler/Task.scala 5 additions, 0 deletionscore/src/main/scala/org/apache/spark/scheduler/Task.scala
- core/src/main/scala/org/apache/spark/util/TaskCompletionListenerException.scala 0 additions, 34 deletions...g/apache/spark/util/TaskCompletionListenerException.scala
- core/src/main/scala/org/apache/spark/util/taskListeners.scala 36 additions, 1 deletion.../src/main/scala/org/apache/spark/util/taskListeners.scala
- core/src/test/java/test/org/apache/spark/JavaTaskCompletionListenerImpl.java 0 additions, 39 deletions...test/org/apache/spark/JavaTaskCompletionListenerImpl.java
- core/src/test/java/test/org/apache/spark/JavaTaskContextCompileCheck.java 30 additions, 0 deletions...va/test/org/apache/spark/JavaTaskContextCompileCheck.java
- core/src/test/scala/org/apache/spark/scheduler/TaskContextSuite.scala 43 additions, 1 deletion...t/scala/org/apache/spark/scheduler/TaskContextSuite.scala
- project/MimaExcludes.scala 3 additions, 1 deletionproject/MimaExcludes.scala
Loading
Please register or sign in to comment