-
- Downloads
Bug fixes + updated limit connecitons shuffle strategy to handle endgame situation.
After the last commit the bottleneck shifted from "requesting to tracker for mappers" (now done in batches) to "notifying tracker when threads leave" (done individually)
Showing
- conf/java-opts 1 addition, 1 deletionconf/java-opts
- src/scala/spark/ShuffleTrackerStrategy.scala 20 additions, 7 deletionssrc/scala/spark/ShuffleTrackerStrategy.scala
- src/scala/spark/TrackedCustomBlockedInMemoryShuffle.scala 2 additions, 1 deletionsrc/scala/spark/TrackedCustomBlockedInMemoryShuffle.scala
- src/scala/spark/TrackedCustomBlockedLocalFileShuffle.scala 2 additions, 1 deletionsrc/scala/spark/TrackedCustomBlockedLocalFileShuffle.scala
- src/scala/spark/TrackedCustomParallelLocalFileShuffle.scala 2 additions, 1 deletionsrc/scala/spark/TrackedCustomParallelLocalFileShuffle.scala
Loading
Please register or sign in to comment