From 4bcb7d676440dedff737a10c98c308d8f2ed1c96 Mon Sep 17 00:00:00 2001
From: jerryshao <sshao@hortonworks.com>
Date: Tue, 28 Mar 2017 10:41:11 -0700
Subject: [PATCH] [SPARK-19995][YARN] Register tokens to current UGI to avoid
re-issuing of tokens in yarn client mode
## What changes were proposed in this pull request?
In the current Spark on YARN code, we will obtain tokens from provided services, but we're not going to add these tokens to the current user's credentials. This will make all the following operations to these services still require TGT rather than delegation tokens. This is unnecessary since we already got the tokens, also this will lead to failure in user impersonation scenario, because the TGT is granted by real user, not proxy user.
So here changing to put all the tokens to the current UGI, so that following operations to these services will honor tokens rather than TGT, and this will further handle the proxy user issue mentioned above.
## How was this patch tested?
Local verified in secure cluster.
vanzin tgravescs mridulm dongjoon-hyun please help to review, thanks a lot.
Author: jerryshao <sshao@hortonworks.com>
Closes #17335 from jerryshao/SPARK-19995.
(cherry picked from commit 17eddb35a280e77da7520343e0bf2a86b329ed62)
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
---
yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala | 3 +++
1 file changed, 3 insertions(+)
diff --git a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
index 5280c420b9..1ba736b221 100644
--- a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
+++ b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
@@ -403,6 +403,9 @@ private[spark] class Client(
val nearestTimeOfNextRenewal = credentialManager.obtainCredentials(hadoopConf, credentials)
if (credentials != null) {
+ // Add credentials to current user's UGI, so that following operations don't need to use the
+ // Kerberos tgt to get delegations again in the client side.
+ UserGroupInformation.getCurrentUser.addCredentials(credentials)
logDebug(YarnSparkHadoopUtil.get.dumpTokens(credentials).mkString("\n"))
}
--
GitLab