-
- Downloads
[SPARK-2503] Lower shuffle output buffer (spark.shuffle.file.buffer.kb) to 32KB.
This can substantially reduce memory usage during shuffle. Author: Reynold Xin <rxin@apache.org> Closes #1781 from rxin/SPARK-2503-spark.shuffle.file.buffer.kb and squashes the following commits: 104b8d8 [Reynold Xin] [SPARK-2503] Lower shuffle output buffer (spark.shuffle.file.buffer.kb) to 32KB.
Showing
- core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleWriter.scala 1 addition, 1 deletion...ala/org/apache/spark/shuffle/sort/SortShuffleWriter.scala
- core/src/main/scala/org/apache/spark/storage/ShuffleBlockManager.scala 1 addition, 1 deletion.../scala/org/apache/spark/storage/ShuffleBlockManager.scala
- core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala 1 addition, 1 deletion.../apache/spark/util/collection/ExternalAppendOnlyMap.scala
- core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala 1 addition, 1 deletion...ala/org/apache/spark/util/collection/ExternalSorter.scala
- docs/configuration.md 1 addition, 1 deletiondocs/configuration.md
Loading
Please register or sign in to comment