Skip to content
Snippets Groups Projects
Commit ddc5baf1 authored by Neelesh Srinivas Salian's avatar Neelesh Srinivas Salian Committed by Davies Liu
Browse files

[SPARK-8320] [STREAMING] Add example in streaming programming guide that shows...

[SPARK-8320] [STREAMING] Add example in streaming programming guide that shows union of multiple input streams

Added python code to https://spark.apache.org/docs/latest/streaming-programming-guide.html
to the Level of Parallelism in Data Receiving section.

Please review and let me know if there are any additional changes that are needed.

Thank you.

Author: Neelesh Srinivas Salian <nsalian@cloudera.com>

Closes #6862 from nssalian/SPARK-8320 and squashes the following commits:

4bfd126 [Neelesh Srinivas Salian] Changed loop structure to be more in line with Python style
e5345de [Neelesh Srinivas Salian] Changes to kafak append, for loop and show to print()
3fc5c6d [Neelesh Srinivas Salian] SPARK-8320
parent e86fbdb1
No related branches found
No related tags found
No related merge requests found
......@@ -1937,6 +1937,14 @@ JavaPairDStream<String, String> unifiedStream = streamingContext.union(kafkaStre
unifiedStream.print();
{% endhighlight %}
</div>
<div data-lang="python" markdown="1">
{% highlight python %}
numStreams = 5
kafkaStreams = [KafkaUtils.createStream(...) for _ in range (numStreams)]
unifiedStream = streamingContext.union(kafkaStreams)
unifiedStream.print()
{% endhighlight %}
</div>
</div>
Another parameter that should be considered is the receiver's blocking interval,
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment