-
- Downloads
[SPARK-10985][CORE] Avoid passing evicted blocks throughout BlockManager
This patch refactors portions of the BlockManager and CacheManager in order to avoid having to pass `evictedBlocks` lists throughout the code. It appears that these lists were only consumed by `TaskContext.taskMetrics`, so the new code now directly updates the metrics from the lower-level BlockManager methods. Author: Josh Rosen <joshrosen@databricks.com> Closes #10776 from JoshRosen/SPARK-10985.
Showing
- core/src/main/scala/org/apache/spark/CacheManager.scala 6 additions, 14 deletionscore/src/main/scala/org/apache/spark/CacheManager.scala
- core/src/main/scala/org/apache/spark/memory/MemoryManager.scala 3 additions, 15 deletions...rc/main/scala/org/apache/spark/memory/MemoryManager.scala
- core/src/main/scala/org/apache/spark/memory/StaticMemoryManager.scala 5 additions, 13 deletions...n/scala/org/apache/spark/memory/StaticMemoryManager.scala
- core/src/main/scala/org/apache/spark/memory/StorageMemoryPool.scala 7 additions, 23 deletions...ain/scala/org/apache/spark/memory/StorageMemoryPool.scala
- core/src/main/scala/org/apache/spark/memory/UnifiedMemoryManager.scala 5 additions, 13 deletions.../scala/org/apache/spark/memory/UnifiedMemoryManager.scala
- core/src/main/scala/org/apache/spark/storage/BlockManager.scala 38 additions, 33 deletions...rc/main/scala/org/apache/spark/storage/BlockManager.scala
- core/src/main/scala/org/apache/spark/storage/MemoryStore.scala 27 additions, 50 deletions...src/main/scala/org/apache/spark/storage/MemoryStore.scala
- core/src/test/scala/org/apache/spark/CacheManagerSuite.scala 7 additions, 2 deletionscore/src/test/scala/org/apache/spark/CacheManagerSuite.scala
- core/src/test/scala/org/apache/spark/memory/MemoryManagerSuite.scala 7 additions, 16 deletions...st/scala/org/apache/spark/memory/MemoryManagerSuite.scala
- core/src/test/scala/org/apache/spark/memory/StaticMemoryManagerSuite.scala 12 additions, 12 deletions...la/org/apache/spark/memory/StaticMemoryManagerSuite.scala
- core/src/test/scala/org/apache/spark/memory/TestMemoryManager.scala 2 additions, 8 deletions...est/scala/org/apache/spark/memory/TestMemoryManager.scala
- core/src/test/scala/org/apache/spark/memory/UnifiedMemoryManagerSuite.scala 15 additions, 15 deletions...a/org/apache/spark/memory/UnifiedMemoryManagerSuite.scala
- core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala 32 additions, 23 deletions...st/scala/org/apache/spark/storage/BlockManagerSuite.scala
- streaming/src/main/scala/org/apache/spark/streaming/receiver/ReceivedBlockHandler.scala 4 additions, 4 deletions...pache/spark/streaming/receiver/ReceivedBlockHandler.scala
Loading
Please register or sign in to comment