-
- Downloads
[SPARK-19960][CORE] Move `SparkHadoopWriter` to `internal/io/`
## What changes were proposed in this pull request? This PR introduces the following changes: 1. Move `SparkHadoopWriter` to `core/internal/io/`, so that it's in the same directory with `SparkHadoopMapReduceWriter`; 2. Move `SparkHadoopWriterUtils` to a separated file. After this PR is merged, we may consolidate `SparkHadoopWriter` and `SparkHadoopMapReduceWriter`, and make the new commit protocol support the old `mapred` package's committer; ## How was this patch tested? Tested by existing test cases. Author: jiangxingbo <jiangxb1987@gmail.com> Closes #17304 from jiangxb1987/writer.
Showing
- core/src/main/scala/org/apache/spark/internal/io/SparkHadoopMapReduceWriter.scala 0 additions, 59 deletions...apache/spark/internal/io/SparkHadoopMapReduceWriter.scala
- core/src/main/scala/org/apache/spark/internal/io/SparkHadoopWriter.scala 3 additions, 4 deletions...cala/org/apache/spark/internal/io/SparkHadoopWriter.scala
- core/src/main/scala/org/apache/spark/internal/io/SparkHadoopWriterUtils.scala 93 additions, 0 deletions...org/apache/spark/internal/io/SparkHadoopWriterUtils.scala
- core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala 2 additions, 1 deletion...rc/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala
- core/src/test/scala/org/apache/spark/scheduler/OutputCommitCoordinatorSuite.scala 1 addition, 0 deletions...apache/spark/scheduler/OutputCommitCoordinatorSuite.scala
Loading
Please register or sign in to comment