Skip to content
Snippets Groups Projects
Commit a26afd52 authored by Shubham Chopra's avatar Shubham Chopra Committed by Reynold Xin
Browse files

[SPARK-15353][CORE] Making peer selection for block replication pluggable

## What changes were proposed in this pull request?

This PR makes block replication strategies pluggable. It provides two trait that can be implemented, one that maps a host to its topology and is used in the master, and the second that helps prioritize a list of peers for block replication and would run in the executors.

This patch contains default implementations of these traits that make sure current Spark behavior is unchanged.

## How was this patch tested?

This patch should not change Spark behavior in any way, and was tested with unit tests for storage.

Author: Shubham Chopra <schopra31@bloomberg.net>

Closes #13152 from shubhamchopra/RackAwareBlockReplication.
parent 81455a9c
No related branches found
No related tags found
No related merge requests found
Showing
with 492 additions and 99 deletions
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment