Skip to content
Snippets Groups Projects
Commit 28f8b721 authored by Tathagata Das's avatar Tathagata Das
Browse files

Added back the initial spark job before starting streaming receivers

parent f282bc49
No related branches found
No related tags found
No related merge requests found
......@@ -141,7 +141,7 @@ class NetworkInputTracker(
}
// Run the dummy Spark job to ensure that all slaves have registered.
// This avoids all the receivers to be scheduled on the same node.
//ssc.sparkContext.makeRDD(1 to 100, 100).map(x => (x, 1)).reduceByKey(_ + _, 20).collect()
ssc.sparkContext.makeRDD(1 to 50, 50).map(x => (x, 1)).reduceByKey(_ + _, 20).collect()
// Distribute the receivers and start them
ssc.sparkContext.runJob(tempRDD, startReceiver)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment