Skip to content
Snippets Groups Projects
Commit e7ec815d authored by Reynold Xin's avatar Reynold Xin
Browse files

Added LZ4 to compression codec in configuration page.

Author: Reynold Xin <rxin@apache.org>

Closes #1417 from rxin/lz4 and squashes the following commits:

472f6a1 [Reynold Xin] Set the proper default.
9cf0b2f [Reynold Xin] Added LZ4 to compression codec in configuration page.
parent 72ea56da
No related branches found
No related tags found
No related merge requests found
...@@ -336,13 +336,12 @@ Apart from these, the following properties are also available, and may be useful ...@@ -336,13 +336,12 @@ Apart from these, the following properties are also available, and may be useful
</tr> </tr>
<tr> <tr>
<td><code>spark.io.compression.codec</code></td> <td><code>spark.io.compression.codec</code></td>
<td>org.apache.spark.io.<br />LZFCompressionCodec</td> <td>org.apache.spark.io.<br />SnappyCompressionCodec</td>
<td> <td>
The codec used to compress internal data such as RDD partitions and shuffle outputs. The codec used to compress internal data such as RDD partitions and shuffle outputs.
By default, Spark provides two codecs: <code>org.apache.spark.io.LZFCompressionCodec</code> By default, Spark provides three codecs: <code>org.apache.spark.io.LZ4CompressionCodec</code>,
and <code>org.apache.spark.io.SnappyCompressionCodec</code>. Of these two choices, <code>org.apache.spark.io.LZFCompressionCodec</code>,
Snappy offers faster compression and decompression, while LZF offers a better compression and <code>org.apache.spark.io.SnappyCompressionCodec</code>.
ratio.
</td> </td>
</tr> </tr>
<tr> <tr>
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment