-
- Downloads
[SPARK-11078] Ensure spilling tests actually spill
#9084 uncovered that many tests that test spilling don't actually spill. This is a follow-up patch to fix that to ensure our unit tests actually catch potential bugs in spilling. The size of this patch is inflated by the refactoring of `ExternalSorterSuite`, which had a lot of duplicate code and logic. Author: Andrew Or <andrew@databricks.com> Closes #9124 from andrewor14/spilling-tests.
Showing
- core/src/main/scala/org/apache/spark/TestUtils.scala 51 additions, 0 deletionscore/src/main/scala/org/apache/spark/TestUtils.scala
- core/src/main/scala/org/apache/spark/shuffle/ShuffleMemoryManager.scala 4 additions, 2 deletions...scala/org/apache/spark/shuffle/ShuffleMemoryManager.scala
- core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala 6 additions, 0 deletions.../apache/spark/util/collection/ExternalAppendOnlyMap.scala
- core/src/main/scala/org/apache/spark/util/collection/Spillable.scala 21 additions, 16 deletions...in/scala/org/apache/spark/util/collection/Spillable.scala
- core/src/test/scala/org/apache/spark/DistributedSuite.scala 26 additions, 13 deletionscore/src/test/scala/org/apache/spark/DistributedSuite.scala
- core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala 59 additions, 44 deletions...he/spark/util/collection/ExternalAppendOnlyMapSuite.scala
- core/src/test/scala/org/apache/spark/util/collection/ExternalSorterSuite.scala 365 additions, 506 deletions...rg/apache/spark/util/collection/ExternalSorterSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/TestShuffleMemoryManager.scala 2 additions, 0 deletions...apache/spark/sql/execution/TestShuffleMemoryManager.scala
Loading
Please register or sign in to comment