-
- Downloads
[SPARK-20548][FLAKY-TEST] share one REPL instance among REPL test cases
## What changes were proposed in this pull request? `ReplSuite.newProductSeqEncoder with REPL defined class` was flaky and throws OOM exception frequently. By analyzing the heap dump, we found the reason is that, in each test case of `ReplSuite`, we create a REPL instance, which creates a classloader and loads a lot of classes related to `SparkContext`. More details please see https://github.com/apache/spark/pull/17833#issuecomment-298711435. In this PR, we create a new test suite, `SingletonReplSuite`, which shares one REPL instances among all the test cases. Then we move most of the tests from `ReplSuite` to `SingletonReplSuite`, to avoid creating a lot of REPL instances and reduce memory footprint. ## How was this patch tested? test only change Author: Wenchen Fan <wenchen@databricks.com> Closes #17844 from cloud-fan/flaky-test.
Showing
- repl/scala-2.11/src/main/scala/org/apache/spark/repl/Main.scala 1 addition, 1 deletion...cala-2.11/src/main/scala/org/apache/spark/repl/Main.scala
- repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoop.scala 1 addition, 8 deletions....11/src/main/scala/org/apache/spark/repl/SparkILoop.scala
- repl/scala-2.11/src/test/scala/org/apache/spark/repl/ReplSuite.scala 2 additions, 270 deletions...2.11/src/test/scala/org/apache/spark/repl/ReplSuite.scala
- repl/scala-2.11/src/test/scala/org/apache/spark/repl/SingletonReplSuite.scala 408 additions, 0 deletions...test/scala/org/apache/spark/repl/SingletonReplSuite.scala
Loading
Please register or sign in to comment