-
- Downloads
[SPARK-2977] Ensure ShuffleManager is created before ShuffleBlockManager
This is intended to fix SPARK-2977. Before, there was an implicit ordering dependency where we needed to know the ShuffleManager implementation before creating the ShuffleBlockManager. This patch makes that dependency explicit by adding ShuffleManager to a bunch of constructors. I think it's a little odd for BlockManager to take a ShuffleManager only to pass it to ShuffleBlockManager without using it itself; there's an opportunity to clean this up later if we sever the circular dependencies between BlockManager and other components and pass those components to BlockManager's constructor. Author: Josh Rosen <joshrosen@apache.org> Closes #1976 from JoshRosen/SPARK-2977 and squashes the following commits: a9cd1e1 [Josh Rosen] [SPARK-2977] Ensure ShuffleManager is created before ShuffleBlockManager.
Showing
- core/src/main/scala/org/apache/spark/SparkEnv.scala 11 additions, 11 deletionscore/src/main/scala/org/apache/spark/SparkEnv.scala
- core/src/main/scala/org/apache/spark/storage/BlockManager.scala 7 additions, 4 deletions...rc/main/scala/org/apache/spark/storage/BlockManager.scala
- core/src/main/scala/org/apache/spark/storage/ShuffleBlockManager.scala 4 additions, 3 deletions.../scala/org/apache/spark/storage/ShuffleBlockManager.scala
- core/src/main/scala/org/apache/spark/storage/ThreadingTest.scala 2 additions, 1 deletion...c/main/scala/org/apache/spark/storage/ThreadingTest.scala
- core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala 7 additions, 5 deletions...st/scala/org/apache/spark/storage/BlockManagerSuite.scala
- core/src/test/scala/org/apache/spark/storage/DiskBlockManagerSuite.scala 6 additions, 2 deletions...cala/org/apache/spark/storage/DiskBlockManagerSuite.scala
Loading
Please register or sign in to comment