-
- Downloads
[SPARK-11195][CORE] Use correct classloader for TaskResultGetter
Make sure we are using the context classloader when deserializing failed TaskResults instead of the Spark classloader. The issue is that `enqueueFailedTask` was using the incorrect classloader which results in `ClassNotFoundException`. Adds a test in TaskResultGetterSuite that compiles a custom exception, throws it on the executor, and asserts that Spark handles the TaskResult deserialization instead of returning `UnknownReason`. See #9367 for previous comments See SPARK-11195 for a full repro Author: Hurshal Patel <hpatel516@gmail.com> Closes #9779 from choochootrain/spark-11195-master.
Showing
- core/src/main/scala/org/apache/spark/TestUtils.scala 6 additions, 5 deletionscore/src/main/scala/org/apache/spark/TestUtils.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskResultGetter.scala 2 additions, 2 deletions...n/scala/org/apache/spark/scheduler/TaskResultGetter.scala
- core/src/test/scala/org/apache/spark/scheduler/TaskResultGetterSuite.scala 64 additions, 1 deletion...la/org/apache/spark/scheduler/TaskResultGetterSuite.scala
Loading
Please register or sign in to comment