Skip to content
Snippets Groups Projects
Commit 9baf0930 authored by Lianhui Wang's avatar Lianhui Wang Committed by Andrew Or
Browse files

[SPARK-8430] ExternalShuffleBlockResolver of shuffle service should support UnsafeShuffleManager

andrewor14 can you take a look?thanks

Author: Lianhui Wang <lianhuiwang09@gmail.com>

Closes #6873 from lianhuiwang/SPARK-8430 and squashes the following commits:

51c47ca [Lianhui Wang] update andrewor's comments
2b27b19 [Lianhui Wang] support UnsafeShuffleManager
parent 2c59d5c1
No related branches found
No related tags found
No related merge requests found
......@@ -108,7 +108,8 @@ public class ExternalShuffleBlockResolver {
if ("org.apache.spark.shuffle.hash.HashShuffleManager".equals(executor.shuffleManager)) {
return getHashBasedShuffleBlockData(executor, blockId);
} else if ("org.apache.spark.shuffle.sort.SortShuffleManager".equals(executor.shuffleManager)) {
} else if ("org.apache.spark.shuffle.sort.SortShuffleManager".equals(executor.shuffleManager)
|| "org.apache.spark.shuffle.unsafe.UnsafeShuffleManager".equals(executor.shuffleManager)) {
return getSortBasedShuffleBlockData(executor, shuffleId, mapId, reduceId);
} else {
throw new UnsupportedOperationException(
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment