Skip to content
Snippets Groups Projects
  • jerryshao's avatar
    9e4e82b7
    [SPARK-5946] [STREAMING] Add Python API for direct Kafka stream · 9e4e82b7
    jerryshao authored
    Currently only added `createDirectStream` API, I'm not sure if `createRDD` is also needed, since some Java object needs to be wrapped in Python. Please help to review, thanks a lot.
    
    Author: jerryshao <saisai.shao@intel.com>
    Author: Saisai Shao <saisai.shao@intel.com>
    
    Closes #4723 from jerryshao/direct-kafka-python-api and squashes the following commits:
    
    a1fe97c [jerryshao] Fix rebase issue
    eebf333 [jerryshao] Address the comments
    da40f4e [jerryshao] Fix Python 2.6 Syntax error issue
    5c0ee85 [jerryshao] Style fix
    4aeac18 [jerryshao] Fix bug in example code
    7146d86 [jerryshao] Add unit test
    bf3bdd6 [jerryshao] Add more APIs and address the comments
    f5b3801 [jerryshao] Small style fix
    8641835 [Saisai Shao] Rebase and update the code
    589c05b [Saisai Shao] Fix the style
    d6fcb6a [Saisai Shao] Address the comments
    dfda902 [Saisai Shao] Style fix
    0f7d168 [Saisai Shao] Add the doc and fix some style issues
    67e6880 [Saisai Shao] Fix test bug
    917b0db [Saisai Shao] Add Python createRDD API for Kakfa direct stream
    c3fc11d [jerryshao] Modify the docs
    2c00936 [Saisai Shao] address the comments
    3360f44 [jerryshao] Fix code style
    e0e0f0d [jerryshao] Code clean and bug fix
    338c41f [Saisai Shao] Add python API and example for direct kafka stream
    9e4e82b7
    History
    [SPARK-5946] [STREAMING] Add Python API for direct Kafka stream
    jerryshao authored
    Currently only added `createDirectStream` API, I'm not sure if `createRDD` is also needed, since some Java object needs to be wrapped in Python. Please help to review, thanks a lot.
    
    Author: jerryshao <saisai.shao@intel.com>
    Author: Saisai Shao <saisai.shao@intel.com>
    
    Closes #4723 from jerryshao/direct-kafka-python-api and squashes the following commits:
    
    a1fe97c [jerryshao] Fix rebase issue
    eebf333 [jerryshao] Address the comments
    da40f4e [jerryshao] Fix Python 2.6 Syntax error issue
    5c0ee85 [jerryshao] Style fix
    4aeac18 [jerryshao] Fix bug in example code
    7146d86 [jerryshao] Add unit test
    bf3bdd6 [jerryshao] Add more APIs and address the comments
    f5b3801 [jerryshao] Small style fix
    8641835 [Saisai Shao] Rebase and update the code
    589c05b [Saisai Shao] Fix the style
    d6fcb6a [Saisai Shao] Address the comments
    dfda902 [Saisai Shao] Style fix
    0f7d168 [Saisai Shao] Add the doc and fix some style issues
    67e6880 [Saisai Shao] Fix test bug
    917b0db [Saisai Shao] Add Python createRDD API for Kakfa direct stream
    c3fc11d [jerryshao] Modify the docs
    2c00936 [Saisai Shao] address the comments
    3360f44 [jerryshao] Fix code style
    e0e0f0d [jerryshao] Code clean and bug fix
    338c41f [Saisai Shao] Add python API and example for direct kafka stream