-
- Downloads
This commit introduces the OpenHashSet and OpenHashMap as indexing primitives.
Large parts of the VertexSetRDD were restructured to take advantage of: 1) the OpenHashSet as an index map 2) view based lazy mapValues and mapValuesWithVertices 3) the cogroup code is currently disabled (since it is not used in any of the tests) The GraphImpl was updated to also use the OpenHashSet and PrimitiveOpenHashMap wherever possible: 1) the LocalVidMaps (used to track replicated vertices) are now implemented using the OpenHashSet 2) an OpenHashMap is temporarily constructed to combine the local OpenHashSet with the local (replicated) vertex attribute arrays 3) because the OpenHashSet constructor grabs a class manifest all operations that construct OpenHashSets have been moved to the GraphImpl Singleton to prevent implicit variable capture within closures.
Showing
- graph/src/main/scala/org/apache/spark/graph/GraphKryoRegistrator.scala 1 addition, 5 deletions...n/scala/org/apache/spark/graph/GraphKryoRegistrator.scala
- graph/src/main/scala/org/apache/spark/graph/VertexSetRDD.scala 98 additions, 116 deletions.../src/main/scala/org/apache/spark/graph/VertexSetRDD.scala
- graph/src/main/scala/org/apache/spark/graph/impl/GraphImpl.scala 113 additions, 104 deletions...rc/main/scala/org/apache/spark/graph/impl/GraphImpl.scala
- graph/src/main/scala/org/apache/spark/graph/package.scala 8 additions, 2 deletionsgraph/src/main/scala/org/apache/spark/graph/package.scala
- graph/src/test/scala/org/apache/spark/graph/GraphSuite.scala 5 additions, 6 deletionsgraph/src/test/scala/org/apache/spark/graph/GraphSuite.scala
Loading
Please register or sign in to comment