Skip to content
Snippets Groups Projects
Commit fef27b29 authored by jerryshao's avatar jerryshao Committed by Tathagata Das
Browse files

[SPARK-4381][Streaming]Add warning log when user set spark.master to local in...

[SPARK-4381][Streaming]Add warning log when user set spark.master to local in Spark Streaming and there's no job executed

Author: jerryshao <saisai.shao@intel.com>

Closes #3244 from jerryshao/SPARK-4381 and squashes the following commits:

d2486c7 [jerryshao] Improve the warning log
d726e85 [jerryshao] Add local[1] to the filter condition
eca428b [jerryshao] Add warning log
parent a51118a3
No related branches found
No related tags found
No related merge requests found
......@@ -121,6 +121,11 @@ class StreamingContext private[streaming] (
}
}
if (sc.conf.get("spark.master") == "local" || sc.conf.get("spark.master") == "local[1]") {
logWarning("spark.master should be set as local[n], n > 1 in local mode if you have receivers" +
" to get data, otherwise Spark jobs will not get resources to process the received data.")
}
private[streaming] val conf = sc.conf
private[streaming] val env = SparkEnv.get
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment