Skip to content
Snippets Groups Projects
  • Tathagata Das's avatar
    b004150a
    [SPARK-4806] Streaming doc update for 1.2 · b004150a
    Tathagata Das authored
    Important updates to the streaming programming guide
    - Make the fault-tolerance properties easier to understand, with information about write ahead logs
    - Update the information about deploying the spark streaming app with information about Driver HA
    - Update Receiver guide to discuss reliable vs unreliable receivers.
    
    Author: Tathagata Das <tathagata.das1565@gmail.com>
    Author: Josh Rosen <joshrosen@databricks.com>
    Author: Josh Rosen <rosenville@gmail.com>
    
    Closes #3653 from tdas/streaming-doc-update-1.2 and squashes the following commits:
    
    f53154a [Tathagata Das] Addressed Josh's comments.
    ce299e4 [Tathagata Das] Minor update.
    ca19078 [Tathagata Das] Minor change
    f746951 [Tathagata Das] Mentioned performance problem with WAL
    7787209 [Tathagata Das] Merge branch 'streaming-doc-update-1.2' of github.com:tdas/spark into streaming-doc-update-1.2
    2184729 [Tathagata Das] Updated Kafka and Flume guides with reliability information.
    2f3178c [Tathagata Das] Added more information about writing reliable receivers in the custom receiver guide.
    91aa5aa [Tathagata Das] Improved API Docs menu
    5707581 [Tathagata Das] Added Pythn API badge
    b9c8c24 [Tathagata Das] Merge pull request #26 from JoshRosen/streaming-programming-guide
    b8c8382 [Josh Rosen] minor fixes
    a4ef126 [Josh Rosen] Restructure parts of the fault-tolerance section to read a bit nicer when skipping over the headings
    65f66cd [Josh Rosen] Fix broken link to fault-tolerance semantics section.
    f015397 [Josh Rosen] Minor grammar / pluralization fixes.
    3019f3a [Josh Rosen] Fix minor Markdown formatting issues
    aa8bb87 [Tathagata Das] Small update.
    195852c [Tathagata Das] Updated based on Josh's comments, updated receiver reliability and deploying section, and also updated configuration.
    17b99fb [Tathagata Das] Merge remote-tracking branch 'apache-github/master' into streaming-doc-update-1.2
    a0217c0 [Tathagata Das] Changed Deploying menu layout
    67fcffc [Tathagata Das] Added cluster mode + supervise example to submitting application guide.
    e45453b [Tathagata Das] Update streaming guide, added deploying section.
    192c7a7 [Tathagata Das] Added more info about Python API, and rewrote the checkpointing section.
    b004150a
    History
    [SPARK-4806] Streaming doc update for 1.2
    Tathagata Das authored
    Important updates to the streaming programming guide
    - Make the fault-tolerance properties easier to understand, with information about write ahead logs
    - Update the information about deploying the spark streaming app with information about Driver HA
    - Update Receiver guide to discuss reliable vs unreliable receivers.
    
    Author: Tathagata Das <tathagata.das1565@gmail.com>
    Author: Josh Rosen <joshrosen@databricks.com>
    Author: Josh Rosen <rosenville@gmail.com>
    
    Closes #3653 from tdas/streaming-doc-update-1.2 and squashes the following commits:
    
    f53154a [Tathagata Das] Addressed Josh's comments.
    ce299e4 [Tathagata Das] Minor update.
    ca19078 [Tathagata Das] Minor change
    f746951 [Tathagata Das] Mentioned performance problem with WAL
    7787209 [Tathagata Das] Merge branch 'streaming-doc-update-1.2' of github.com:tdas/spark into streaming-doc-update-1.2
    2184729 [Tathagata Das] Updated Kafka and Flume guides with reliability information.
    2f3178c [Tathagata Das] Added more information about writing reliable receivers in the custom receiver guide.
    91aa5aa [Tathagata Das] Improved API Docs menu
    5707581 [Tathagata Das] Added Pythn API badge
    b9c8c24 [Tathagata Das] Merge pull request #26 from JoshRosen/streaming-programming-guide
    b8c8382 [Josh Rosen] minor fixes
    a4ef126 [Josh Rosen] Restructure parts of the fault-tolerance section to read a bit nicer when skipping over the headings
    65f66cd [Josh Rosen] Fix broken link to fault-tolerance semantics section.
    f015397 [Josh Rosen] Minor grammar / pluralization fixes.
    3019f3a [Josh Rosen] Fix minor Markdown formatting issues
    aa8bb87 [Tathagata Das] Small update.
    195852c [Tathagata Das] Updated based on Josh's comments, updated receiver reliability and deploying section, and also updated configuration.
    17b99fb [Tathagata Das] Merge remote-tracking branch 'apache-github/master' into streaming-doc-update-1.2
    a0217c0 [Tathagata Das] Changed Deploying menu layout
    67fcffc [Tathagata Das] Added cluster mode + supervise example to submitting application guide.
    e45453b [Tathagata Das] Update streaming guide, added deploying section.
    192c7a7 [Tathagata Das] Added more info about Python API, and rewrote the checkpointing section.