Skip to content
Snippets Groups Projects
Commit 2ef4c596 authored by Davies Liu's avatar Davies Liu Committed by Davies Liu
Browse files

[SPARK-13830] prefer block manager than direct result for large result

## What changes were proposed in this pull request?

The current RPC can't handle large blocks very well, it's very slow to fetch 100M block (about 1 minute). Once switch to block manager to fetch that, it took about 10 seconds (still could be improved).

## How was this patch tested?

existing unit tests.

Author: Davies Liu <davies@databricks.com>

Closes #11659 from davies/direct_result.
parent 66d9d0ed
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment