jinxing
authored
## What changes were proposed in this pull request? When application contains large amount of shuffle blocks. NodeManager requires lots of memory to keep metadata(`FileSegmentManagedBuffer`) in `StreamManager`. When the number of shuffle blocks is big enough. NodeManager can run OOM. This pr proposes to do lazy initialization of `FileSegmentManagedBuffer` in shuffle service. ## How was this patch tested? Manually test. Author: jinxing <jinxing6042@126.com> Closes #17744 from jinxing64/SPARK-20426.
| Name | Last commit | Last update |
|---|---|---|
| .. | ||
| network-common | ||
| network-shuffle | ||
| network-yarn | ||
| sketch | ||
| tags | ||
| unsafe | ||