Skip to content
Snippets Groups Projects
Commit 945fe7a3 authored by Patrick Wendell's avatar Patrick Wendell
Browse files

Merge pull request #408 from pwendell/external-serializers

Improvements to external sorting

1. Adds the option of compressing outputs.
2. Adds batching to the serialization to prevent OOM on the read side.
3. Slight renaming of config options.
4. Use Spark's buffer size for reads in addition to writes.
parents 68641bce d4cd5deb
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment