Skip to content
Snippets Groups Projects
Commit 092121e4 authored by Davies Liu's avatar Davies Liu Committed by Matei Zaharia
Browse files

[SPARK-3239] [PySpark] randomize the dirs for each process 

This can avoid the IO contention during spilling, when you have multiple disks.

Author: Davies Liu <davies.liu@gmail.com>

Closes #2152 from davies/randomize and squashes the following commits:

a4863c4 [Davies Liu] randomize the dirs for each process
parent 8f8e2a4e
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment