From 298c4ae81520b6b39230a6b0bf733c2b7caea627 Mon Sep 17 00:00:00 2001
From: Dhruve Ashar <dhruveashar@gmail.com>
Date: Fri, 17 Jun 2016 15:44:33 -0500
Subject: [PATCH] [SPARK-16018][SHUFFLE] Shade netty to load shuffle jar in
Nodemanger
## What changes were proposed in this pull request?
Shade the netty.io namespace so that we can use it in shuffle independent of the dependencies being pulled by hadoop jars.
## How was this patch tested?
Ran a decent job involving shuffle write/read and tested the new spark-x-yarn-shuffle jar. After shading netty.io namespace, the nodemanager loads and shuffle job completes successfully.
Author: Dhruve Ashar <dhruveashar@gmail.com>
Closes #13739 from dhruve/bug/SPARK-16018.
---
common/network-yarn/pom.xml | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/common/network-yarn/pom.xml b/common/network-yarn/pom.xml
index 1fd3af2e6e..3b7ffe8277 100644
--- a/common/network-yarn/pom.xml
+++ b/common/network-yarn/pom.xml
@@ -96,6 +96,13 @@
<include>com.fasterxml.jackson.**</include>
</includes>
</relocation>
+ <relocation>
+ <pattern>io.netty</pattern>
+ <shadedPattern>${spark.shade.packageName}.io.netty</shadedPattern>
+ <includes>
+ <include>io.netty.**</include>
+ </includes>
+ </relocation>
</relocations>
</configuration>
<executions>
--
GitLab