-
- Downloads
[SPARK-13601] call failure callbacks before writer.close()
## What changes were proposed in this pull request? In order to tell OutputStream that the task has failed or not, we should call the failure callbacks BEFORE calling writer.close(). ## How was this patch tested? Added new unit tests. Author: Davies Liu <davies@databricks.com> Closes #11450 from davies/callback.
Showing
- core/src/main/scala/org/apache/spark/TaskContextImpl.scala 8 additions, 2 deletionscore/src/main/scala/org/apache/spark/TaskContextImpl.scala
- core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala 2 additions, 2 deletions...rc/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala
- core/src/main/scala/org/apache/spark/util/Utils.scala 38 additions, 1 deletioncore/src/main/scala/org/apache/spark/util/Utils.scala
- core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala 90 additions, 1 deletion...st/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/WriterContainer.scala 49 additions, 46 deletions...che/spark/sql/execution/datasources/WriterContainer.scala
- sql/hive/src/test/scala/org/apache/spark/sql/sources/CommitFailureTestRelationSuite.scala 57 additions, 0 deletions...he/spark/sql/sources/CommitFailureTestRelationSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/sources/SimpleTextRelation.scala 27 additions, 1 deletion...ala/org/apache/spark/sql/sources/SimpleTextRelation.scala
Loading
Please register or sign in to comment