Skip to content
Snippets Groups Projects
Commit 5f0cdabd authored by Mosharaf Chowdhury's avatar Mosharaf Chowdhury
Browse files

Added a separate thread to deserialize (1 thread per reducer) in CustomParallelLocalFileShuffle

Upside: No synchronized blocking on "combiners" variable. 3x faster :)
Downside: Inefficient implementation. Requiring too much temporary data. Approx. 2x increase in memory requirement :( Should be fixed at some point.
parent f4d0e917
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment