-
- Downloads
[SPARK-6479] [BLOCK MANAGER] Create off-heap block storage API
This is the classes for creating off-heap block storage API. It also includes the migration for Tachyon. The diff seems to be big, but it mainly just rename tachyon to offheap. New implementation for hdfs will be submit for review in spark-6112. Author: Zhan Zhang <zhazhan@gmail.com> Closes #5430 from zhzhan/SPARK-6479 and squashes the following commits: 60acd84 [Zhan Zhang] minor change to kickoff the test 12f54c9 [Zhan Zhang] solve merge conflicts a54132c [Zhan Zhang] solve review comments ffb8e00 [Zhan Zhang] rebase to sparkcontext change 6e121e0 [Zhan Zhang] resolve review comments and restructure blockmanasger code a7aed6c [Zhan Zhang] add Tachyon migration code 186de31 [Zhan Zhang] initial commit for off-heap block storage api
Showing
- core/src/main/scala/org/apache/spark/SparkContext.scala 5 additions, 3 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/executor/ExecutorExitCode.scala 9 additions, 7 deletions...in/scala/org/apache/spark/executor/ExecutorExitCode.scala
- core/src/main/scala/org/apache/spark/rdd/RDD.scala 2 additions, 2 deletionscore/src/main/scala/org/apache/spark/rdd/RDD.scala
- core/src/main/scala/org/apache/spark/storage/BlockManager.scala 29 additions, 34 deletions...rc/main/scala/org/apache/spark/storage/BlockManager.scala
- core/src/main/scala/org/apache/spark/storage/BlockManagerMaster.scala 3 additions, 2 deletions...n/scala/org/apache/spark/storage/BlockManagerMaster.scala
- core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala 15 additions, 14 deletions...org/apache/spark/storage/BlockManagerMasterEndpoint.scala
- core/src/main/scala/org/apache/spark/storage/BlockManagerMessages.scala 3 additions, 3 deletions...scala/org/apache/spark/storage/BlockManagerMessages.scala
- core/src/main/scala/org/apache/spark/storage/ExternalBlockManager.scala 102 additions, 0 deletions...scala/org/apache/spark/storage/ExternalBlockManager.scala
- core/src/main/scala/org/apache/spark/storage/ExternalBlockStore.scala 181 additions, 0 deletions...n/scala/org/apache/spark/storage/ExternalBlockStore.scala
- core/src/main/scala/org/apache/spark/storage/RDDInfo.scala 5 additions, 4 deletionscore/src/main/scala/org/apache/spark/storage/RDDInfo.scala
- core/src/main/scala/org/apache/spark/storage/StorageLevel.scala 4 additions, 4 deletions...rc/main/scala/org/apache/spark/storage/StorageLevel.scala
- core/src/main/scala/org/apache/spark/storage/StorageUtils.scala 10 additions, 9 deletions...rc/main/scala/org/apache/spark/storage/StorageUtils.scala
- core/src/main/scala/org/apache/spark/storage/TachyonBlockManager.scala 100 additions, 38 deletions.../scala/org/apache/spark/storage/TachyonBlockManager.scala
- core/src/main/scala/org/apache/spark/storage/TachyonFileSegment.scala 0 additions, 30 deletions...n/scala/org/apache/spark/storage/TachyonFileSegment.scala
- core/src/main/scala/org/apache/spark/ui/storage/StoragePage.scala 2 additions, 2 deletions.../main/scala/org/apache/spark/ui/storage/StoragePage.scala
- core/src/main/scala/org/apache/spark/util/JsonProtocol.scala 15 additions, 9 deletionscore/src/main/scala/org/apache/spark/util/JsonProtocol.scala
- core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala 1 addition, 0 deletions...st/scala/org/apache/spark/storage/BlockManagerSuite.scala
- core/src/test/scala/org/apache/spark/storage/StorageSuite.scala 4 additions, 4 deletions...rc/test/scala/org/apache/spark/storage/StorageSuite.scala
- core/src/test/scala/org/apache/spark/ui/storage/StorageTabSuite.scala 3 additions, 3 deletions...t/scala/org/apache/spark/ui/storage/StorageTabSuite.scala
- core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala 28 additions, 28 deletions.../test/scala/org/apache/spark/util/JsonProtocolSuite.scala
Loading
Please register or sign in to comment