Skip to content
Snippets Groups Projects
Commit 0375a413 authored by 许鹏's avatar 许鹏 Committed by Andrew Or
Browse files

fix spark-6033, clarify the spark.worker.cleanup behavior in standalone mode

jira case spark-6033 https://issues.apache.org/jira/browse/SPARK-6033

In standalone deploy mode, the cleanup will only remove the stopped application's directories.

The original description about the cleanup behavior is incorrect.

Author: 许鹏 <peng.xu@fraudmetrix.cn>

Closes #4803 from hseagle/spark-6033 and squashes the following commits:

927a6a0 [许鹏] fix the incorrect description about the spark.worker.cleanup in standalone mode
parent 7c99a014
No related branches found
No related tags found
No related merge requests found
...@@ -222,8 +222,7 @@ SPARK_WORKER_OPTS supports the following system properties: ...@@ -222,8 +222,7 @@ SPARK_WORKER_OPTS supports the following system properties:
<td>false</td> <td>false</td>
<td> <td>
Enable periodic cleanup of worker / application directories. Note that this only affects standalone Enable periodic cleanup of worker / application directories. Note that this only affects standalone
mode, as YARN works differently. Applications directories are cleaned up regardless of whether mode, as YARN works differently. Only the directories of stopped applications are cleaned up.
the application is still running.
</td> </td>
</tr> </tr>
<tr> <tr>
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment