Skip to content
Snippets Groups Projects
Commit f268548d authored by Ankur Dave's avatar Ankur Dave Committed by Reynold Xin
Browse files

[SPARK-1931] Reconstruct routing tables in Graph.partitionBy


905173df introduced a bug in partitionBy where, after repartitioning the edges, it reuses the VertexRDD without updating the routing tables to reflect the new edge layout. Subsequent accesses of the triplets contain nulls for many vertex properties.

This commit adds a test for this bug and fixes it by introducing `VertexRDD#withEdges` and calling it in `partitionBy`.

Author: Ankur Dave <ankurdave@gmail.com>

Closes #885 from ankurdave/SPARK-1931 and squashes the following commits:

3930cdd [Ankur Dave] Note how to set up VertexRDD for efficient joins
9bdbaa4 [Ankur Dave] [SPARK-1931] Reconstruct routing tables in Graph.partitionBy

(cherry picked from commit 56c771cb)
Signed-off-by: default avatarReynold Xin <rxin@apache.org>
parent f09cb850
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment