Skip to content
Snippets Groups Projects
Commit 96fb894d authored by Tom Graves's avatar Tom Graves Committed by Sean Owen
Browse files

[SPARK-2930] clarify docs on using webhdfs with spark.yarn.access.nam…

…enodes

Author: Tom Graves <tgraves@yahoo-inc.com>

Closes #10699 from tgravescs/SPARK-2930.
parent d0a5c32b
No related branches found
No related tags found
No related merge requests found
...@@ -260,10 +260,10 @@ If you need a reference to the proper location to put log files in the YARN so t ...@@ -260,10 +260,10 @@ If you need a reference to the proper location to put log files in the YARN so t
<td>(none)</td> <td>(none)</td>
<td> <td>
A comma-separated list of secure HDFS namenodes your Spark application is going to access. For A comma-separated list of secure HDFS namenodes your Spark application is going to access. For
example, <code>spark.yarn.access.namenodes=hdfs://nn1.com:8032,hdfs://nn2.com:8032</code>. example, <code>spark.yarn.access.namenodes=hdfs://nn1.com:8032,hdfs://nn2.com:8032,
The Spark application must have access to the namenodes listed and Kerberos must webhdfs://nn3.com:50070</code>. The Spark application must have access to the namenodes listed
be properly configured to be able to access them (either in the same realm or in and Kerberos must be properly configured to be able to access them (either in the same realm
a trusted realm). Spark acquires security tokens for each of the namenodes so that or in a trusted realm). Spark acquires security tokens for each of the namenodes so that
the Spark application can access those remote HDFS clusters. the Spark application can access those remote HDFS clusters.
</td> </td>
</tr> </tr>
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment