Skip to content
Snippets Groups Projects
Commit 87676a6a authored by Matei Zaharia's avatar Matei Zaharia
Browse files

Merge pull request #220 from rxin/zippart


Memoize preferred locations in ZippedPartitionsBaseRDD

so preferred location computation doesn't lead to exponential explosion.

This was a problem in GraphX where we have a whole chain of RDDs that are ZippedPartitionsRDD's, and the preferred locations were taking eternity to compute.

(cherry picked from commit e36fe55a)
Signed-off-by: default avatarReynold Xin <rxin@apache.org>
parents 07804987 9cf7f31e
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment