-
- Downloads
[SPARK-4672][GraphX]Non-transient PartitionsRDDs will lead to StackOverflow error
The related JIRA is https://issues.apache.org/jira/browse/SPARK-4672 In a nutshell, if `val partitionsRDD` in EdgeRDDImpl and VertexRDDImpl are non-transient, the serialization chain can become very long in iterative algorithms and finally lead to the StackOverflow error. More details and explanation can be found in the JIRA. Author: JerryLead <JerryLead@163.com> Author: Lijie Xu <csxulijie@gmail.com> Closes #3544 from JerryLead/my_graphX and squashes the following commits: 628f33c [JerryLead] set PartitionsRDD to be transient in EdgeRDDImpl and VertexRDDImpl c0169da [JerryLead] Merge branch 'master' of https://github.com/apache/spark 52799e3 [Lijie Xu] Merge pull request #1 from apache/master
Showing
- graphx/src/main/scala/org/apache/spark/graphx/impl/EdgeRDDImpl.scala 1 addition, 1 deletion...main/scala/org/apache/spark/graphx/impl/EdgeRDDImpl.scala
- graphx/src/main/scala/org/apache/spark/graphx/impl/VertexRDDImpl.scala 1 addition, 1 deletion...in/scala/org/apache/spark/graphx/impl/VertexRDDImpl.scala
Please register or sign in to comment