-
- Downloads
Report errors in tasks to the driver via a Mesos status update
When a task throws an exception, the Spark executor previously just logged it to a local file on the slave and exited. This commit causes Spark to also report the exception back to the driver using a Mesos status update, so the user doesn't have to look through a log file on the slave. Here's what the reporting currently looks like: # ./run spark.examples.ExceptionHandlingTest master@203.0.113.1:5050 [...] 11/10/26 21:04:13 INFO spark.SimpleJob: Lost TID 1 (task 0:1) 11/10/26 21:04:13 INFO spark.SimpleJob: Loss was due to java.lang.Exception: Testing exception handling [...] 11/10/26 21:04:16 INFO spark.SparkContext: Job finished in 5.988547328 s
Showing
- core/src/main/scala/spark/Executor.scala 7 additions, 0 deletionscore/src/main/scala/spark/Executor.scala
- core/src/main/scala/spark/SimpleJob.scala 2 additions, 0 deletionscore/src/main/scala/spark/SimpleJob.scala
- examples/src/main/scala/spark/examples/ExceptionHandlingTest.scala 18 additions, 0 deletions...src/main/scala/spark/examples/ExceptionHandlingTest.scala
Loading
Please register or sign in to comment