-
- Downloads
[SPARK-5946] [STREAMING] Add Python API for direct Kafka stream
Currently only added `createDirectStream` API, I'm not sure if `createRDD` is also needed, since some Java object needs to be wrapped in Python. Please help to review, thanks a lot. Author: jerryshao <saisai.shao@intel.com> Author: Saisai Shao <saisai.shao@intel.com> Closes #4723 from jerryshao/direct-kafka-python-api and squashes the following commits: a1fe97c [jerryshao] Fix rebase issue eebf333 [jerryshao] Address the comments da40f4e [jerryshao] Fix Python 2.6 Syntax error issue 5c0ee85 [jerryshao] Style fix 4aeac18 [jerryshao] Fix bug in example code 7146d86 [jerryshao] Add unit test bf3bdd6 [jerryshao] Add more APIs and address the comments f5b3801 [jerryshao] Small style fix 8641835 [Saisai Shao] Rebase and update the code 589c05b [Saisai Shao] Fix the style d6fcb6a [Saisai Shao] Address the comments dfda902 [Saisai Shao] Style fix 0f7d168 [Saisai Shao] Add the doc and fix some style issues 67e6880 [Saisai Shao] Fix test bug 917b0db [Saisai Shao] Add Python createRDD API for Kakfa direct stream c3fc11d [jerryshao] Modify the docs 2c00936 [Saisai Shao] address the comments 3360f44 [jerryshao] Fix code style e0e0f0d [jerryshao] Code clean and bug fix 338c41f [Saisai Shao] Add python API and example for direct kafka stream
Showing
- examples/src/main/python/streaming/direct_kafka_wordcount.py 55 additions, 0 deletionsexamples/src/main/python/streaming/direct_kafka_wordcount.py
- external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala 91 additions, 1 deletion...n/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
- python/pyspark/streaming/kafka.py 160 additions, 7 deletionspython/pyspark/streaming/kafka.py
- python/pyspark/streaming/tests.py 77 additions, 7 deletionspython/pyspark/streaming/tests.py
Loading
Please register or sign in to comment