From 0d7aac99da660cc42eb5a9be8e262bd9bd8a770f Mon Sep 17 00:00:00 2001
From: Sean Owen <sowen@cloudera.com>
Date: Thu, 6 Aug 2015 19:29:42 +0100
Subject: [PATCH] [SPARK-9641] [DOCS] spark.shuffle.service.port is not
documented
Document spark.shuffle.service.{enabled,port}
CC sryza tgravescs
This is pretty minimal; is there more to say here about the service?
Author: Sean Owen <sowen@cloudera.com>
Closes #7991 from srowen/SPARK-9641 and squashes the following commits:
3bb946e [Sean Owen] Add link to docs for setup and config of external shuffle service
2302e01 [Sean Owen] Document spark.shuffle.service.{enabled,port}
---
docs/configuration.md | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/docs/configuration.md b/docs/configuration.md
index 24b606356a..c60dd16839 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -473,6 +473,25 @@ Apart from these, the following properties are also available, and may be useful
<code>spark.storage.memoryFraction</code>.
</td>
</tr>
+<tr>
+ <td><code>spark.shuffle.service.enabled</code></td>
+ <td>false</td>
+ <td>
+ Enables the external shuffle service. This service preserves the shuffle files written by
+ executors so the executors can be safely removed. This must be enabled if
+ <code>spark.dynamicAllocation.enabled</code> is "true". The external shuffle service
+ must be set up in order to enable it. See
+ <a href="job-scheduling.html#configuration-and-setup">dynamic allocation
+ configuration and setup documentation</a> for more information.
+ </td>
+</tr>
+<tr>
+ <td><code>spark.shuffle.service.port</code></td>
+ <td>7337</td>
+ <td>
+ Port on which the external shuffle service will run.
+ </td>
+</tr>
<tr>
<td><code>spark.shuffle.sort.bypassMergeThreshold</code></td>
<td>200</td>
--
GitLab