-
- Downloads
[SPARK-2125] Add sort flag and move sort into shuffle implementations
This patch adds a sort flag into ShuffleDependecy and moves sort into hash shuffle implementation. Moving sort into shuffle implementation can give space for other shuffle implementations (like sort-based shuffle) to better optimize sort through shuffle. Author: jerryshao <saisai.shao@intel.com> Closes #1210 from jerryshao/SPARK-2125 and squashes the following commits: 2feaf7b [jerryshao] revert MimaExcludes ceddf75 [jerryshao] add MimaExeclude f674ff4 [jerryshao] Add missing Scope restriction b9fe0dd [jerryshao] Fix some style issues according to comments ef6b729 [jerryshao] Change sort flag into Option 3f6eeed [jerryshao] Fix issues related to unit test 2f552a5 [jerryshao] Minor changes about naming and order c92a281 [jerryshao] Move sort into shuffle implementations
Showing
- core/src/main/scala/org/apache/spark/Dependency.scala 3 additions, 1 deletioncore/src/main/scala/org/apache/spark/Dependency.scala
- core/src/main/scala/org/apache/spark/rdd/OrderedRDDFunctions.scala 8 additions, 9 deletions...main/scala/org/apache/spark/rdd/OrderedRDDFunctions.scala
- core/src/main/scala/org/apache/spark/rdd/ShuffledRDD.scala 11 additions, 1 deletioncore/src/main/scala/org/apache/spark/rdd/ShuffledRDD.scala
- core/src/main/scala/org/apache/spark/shuffle/hash/HashShuffleReader.scala 13 additions, 1 deletion...ala/org/apache/spark/shuffle/hash/HashShuffleReader.scala
Loading
Please register or sign in to comment