-
- Downloads
Shuffle Performance fix: Use netty embeded OIO file server instead of ConnectionManager
Shuffle Performance Optimization: do not send 0-byte block requests to reduce network messages
change reference from io.Source to scala.io.Source to avoid looking into io.netty package
Signed-off-by:
shane-huang <shengsheng.huang@intel.com>
Showing
- core/src/main/java/spark/network/netty/FileClient.java 89 additions, 0 deletionscore/src/main/java/spark/network/netty/FileClient.java
- core/src/main/java/spark/network/netty/FileClientChannelInitializer.java 29 additions, 0 deletions...ava/spark/network/netty/FileClientChannelInitializer.java
- core/src/main/java/spark/network/netty/FileClientHandler.java 38 additions, 0 deletions.../src/main/java/spark/network/netty/FileClientHandler.java
- core/src/main/java/spark/network/netty/FileServer.java 59 additions, 0 deletionscore/src/main/java/spark/network/netty/FileServer.java
- core/src/main/java/spark/network/netty/FileServerChannelInitializer.java 33 additions, 0 deletions...ava/spark/network/netty/FileServerChannelInitializer.java
- core/src/main/java/spark/network/netty/FileServerHandler.java 68 additions, 0 deletions.../src/main/java/spark/network/netty/FileServerHandler.java
- core/src/main/java/spark/network/netty/PathResolver.java 12 additions, 0 deletionscore/src/main/java/spark/network/netty/PathResolver.java
- core/src/main/scala/spark/network/netty/FileHeader.scala 57 additions, 0 deletionscore/src/main/scala/spark/network/netty/FileHeader.scala
- core/src/main/scala/spark/network/netty/ShuffleCopier.scala 88 additions, 0 deletionscore/src/main/scala/spark/network/netty/ShuffleCopier.scala
- core/src/main/scala/spark/network/netty/ShuffleSender.scala 50 additions, 0 deletionscore/src/main/scala/spark/network/netty/ShuffleSender.scala
- core/src/main/scala/spark/storage/BlockManager.scala 219 additions, 53 deletionscore/src/main/scala/spark/storage/BlockManager.scala
- core/src/main/scala/spark/storage/DiskStore.scala 50 additions, 1 deletioncore/src/main/scala/spark/storage/DiskStore.scala
- project/SparkBuild.scala 2 additions, 1 deletionproject/SparkBuild.scala
- streaming/src/main/scala/spark/streaming/util/RawTextSender.scala 1 addition, 1 deletion...g/src/main/scala/spark/streaming/util/RawTextSender.scala
Loading
Please register or sign in to comment