Skip to content
Snippets Groups Projects
Commit 298c4ae8 authored by Dhruve Ashar's avatar Dhruve Ashar Committed by Tom Graves
Browse files

[SPARK-16018][SHUFFLE] Shade netty to load shuffle jar in Nodemanger

## What changes were proposed in this pull request?
Shade the netty.io namespace so that we can use it in shuffle independent of the dependencies being pulled by hadoop jars.

## How was this patch tested?
Ran a decent job involving shuffle write/read and tested the new spark-x-yarn-shuffle jar. After shading netty.io namespace, the nodemanager loads and shuffle job completes successfully.

Author: Dhruve Ashar <dhruveashar@gmail.com>

Closes #13739 from dhruve/bug/SPARK-16018.
parent c8809db5
No related branches found
No related tags found
No related merge requests found
......@@ -96,6 +96,13 @@
<include>com.fasterxml.jackson.**</include>
</includes>
</relocation>
<relocation>
<pattern>io.netty</pattern>
<shadedPattern>${spark.shade.packageName}.io.netty</shadedPattern>
<includes>
<include>io.netty.**</include>
</includes>
</relocation>
</relocations>
</configuration>
<executions>
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment