-
- Downloads
SPARK-1772 Stop catching Throwable, let Executors die
The main issue this patch fixes is [SPARK-1772](https://issues.apache.org/jira/browse/SPARK-1772), in which Executors may not die when fatal exceptions (e.g., OOM) are thrown. This patch causes Executors to delegate to the ExecutorUncaughtExceptionHandler when a fatal exception is thrown. This patch also continues the fight in the neverending war against `case t: Throwable =>`, by only catching Exceptions in many places, and adding a wrapper for Threads and Runnables to make sure any uncaught exceptions are at least printed to the logs. It also turns out that it is unlikely that the IndestructibleActorSystem actually works, given testing ([here](https://gist.github.com/aarondav/ca1f0cdcd50727f89c0d)). The uncaughtExceptionHandler is not called from the places that we expected it would be. [SPARK-1620](https://issues.apache.org/jira/browse/SPARK-1620) deals with part of this issue, but refactoring our Actor Systems to ensure that exceptions are dealt with properly is a much bigger change, outside the scope of this PR. Author: Aaron Davidson <aaron@databricks.com> Closes #715 from aarondav/throwable and squashes the following commits: f9b9bfe [Aaron Davidson] Remove other redundant 'throw e' e937a0a [Aaron Davidson] Address Prashant and Matei's comments 1867867 [Aaron Davidson] [RFC] SPARK-1772 Stop catching Throwable, let Executors die
Showing
- core/src/main/scala/org/apache/spark/ContextCleaner.scala 6 additions, 5 deletionscore/src/main/scala/org/apache/spark/ContextCleaner.scala
- core/src/main/scala/org/apache/spark/SparkContext.scala 6 additions, 6 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala 1 addition, 2 deletions...rc/main/scala/org/apache/spark/api/python/PythonRDD.scala
- core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala 0 additions, 1 deletion...ala/org/apache/spark/api/python/PythonWorkerFactory.scala
- core/src/main/scala/org/apache/spark/deploy/Client.scala 1 addition, 1 deletioncore/src/main/scala/org/apache/spark/deploy/Client.scala
- core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala 1 addition, 1 deletion.../main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
- core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala 4 additions, 4 deletions...scala/org/apache/spark/deploy/history/HistoryServer.scala
- core/src/main/scala/org/apache/spark/deploy/master/Master.scala 2 additions, 2 deletions...rc/main/scala/org/apache/spark/deploy/master/Master.scala
- core/src/main/scala/org/apache/spark/deploy/worker/DriverWrapper.scala 1 addition, 1 deletion.../scala/org/apache/spark/deploy/worker/DriverWrapper.scala
- core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala 1 addition, 1 deletion.../apache/spark/executor/CoarseGrainedExecutorBackend.scala
- core/src/main/scala/org/apache/spark/executor/Executor.scala 11 additions, 26 deletionscore/src/main/scala/org/apache/spark/executor/Executor.scala
- core/src/main/scala/org/apache/spark/executor/ExecutorUncaughtExceptionHandler.scala 53 additions, 0 deletions...che/spark/executor/ExecutorUncaughtExceptionHandler.scala
- core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala 2 additions, 2 deletions...ala/org/apache/spark/scheduler/EventLoggingListener.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskResultGetter.scala 4 additions, 4 deletions...n/scala/org/apache/spark/scheduler/TaskResultGetter.scala
- core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala 3 additions, 3 deletions...ain/scala/org/apache/spark/storage/DiskBlockManager.scala
- core/src/main/scala/org/apache/spark/storage/TachyonBlockManager.scala 3 additions, 4 deletions.../scala/org/apache/spark/storage/TachyonBlockManager.scala
- core/src/main/scala/org/apache/spark/util/AkkaUtils.scala 3 additions, 8 deletionscore/src/main/scala/org/apache/spark/util/AkkaUtils.scala
- core/src/main/scala/org/apache/spark/util/Utils.scala 25 additions, 1 deletioncore/src/main/scala/org/apache/spark/util/Utils.scala
Loading
Please register or sign in to comment