Skip to content
Snippets Groups Projects
Commit b84dc780 authored by Prashant Sharma's avatar Prashant Sharma
Browse files

Allow configuration to be printed in logs for diagnosis.

parent b3018811
No related branches found
No related tags found
No related merge requests found
......@@ -116,6 +116,10 @@ class SparkContext(
throw new SparkException("An application must be set in your configuration")
}
if (conf.get("spark.log-conf", "false").toBoolean) {
logInfo("Spark configuration:\n" + conf.toDebugString)
}
// Set Spark driver host and port system properties
conf.setIfMissing("spark.driver.host", Utils.localHostName())
conf.setIfMissing("spark.driver.port", "0")
......
......@@ -403,6 +403,13 @@ Apart from these, the following properties are also available, and may be useful
How many times slower a task is than the median to be considered for speculation.
</td>
</tr>
<tr>
<td>spark.log-conf</td>
<td>false</td>
<td>
Log the supplied SparkConf as INFO at start of spark context.
</td>
</tr>
</table>
## Viewing Spark Properties
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment