Skip to content
Snippets Groups Projects
  1. Sep 29, 2016
    • Takeshi YAMAMURO's avatar
      [MINOR][DOCS] Fix th doc. of spark-streaming with kinesis · b2e9731c
      Takeshi YAMAMURO authored
      ## What changes were proposed in this pull request?
      This pr is just to fix the document of `spark-kinesis-integration`.
      Since `SPARK-17418` prevented all the kinesis stuffs (including kinesis example code)
      from publishing,  `bin/run-example streaming.KinesisWordCountASL` and `bin/run-example streaming.JavaKinesisWordCountASL` does not work.
      Instead, it fetches the kinesis jar from the Spark Package.
      
      Author: Takeshi YAMAMURO <linguin.m.s@gmail.com>
      
      Closes #15260 from maropu/DocFixKinesis.
      Unverified
      b2e9731c
  2. Jul 11, 2016
    • Xin Ren's avatar
      [MINOR][STREAMING][DOCS] Minor changes on kinesis integration · 05d7151c
      Xin Ren authored
      ## What changes were proposed in this pull request?
      
      Some minor changes for documentation page "Spark Streaming + Kinesis Integration".
      
      Moved "streaming-kinesis-arch.png" before the bullet list, not in between the bullets.
      
      ## How was this patch tested?
      
      Tested manually, on my local machine.
      
      Author: Xin Ren <iamshrek@126.com>
      
      Closes #14097 from keypointt/kinesisDoc.
      05d7151c
  3. Mar 09, 2016
    • Sean Owen's avatar
      [SPARK-13595][BUILD] Move docker, extras modules into external · 256704c7
      Sean Owen authored
      ## What changes were proposed in this pull request?
      
      Move `docker` dirs out of top level into `external/`; move `extras/*` into `external/`
      
      ## How was this patch tested?
      
      This is tested with Jenkins tests.
      
      Author: Sean Owen <sowen@cloudera.com>
      
      Closes #11523 from srowen/SPARK-13595.
      256704c7
  4. Feb 22, 2016
  5. Jan 18, 2016
  6. Dec 18, 2015
  7. Aug 28, 2015
  8. Jul 31, 2015
    • zsxwing's avatar
      [SPARK-8564] [STREAMING] Add the Python API for Kinesis · 3afc1de8
      zsxwing authored
      This PR adds the Python API for Kinesis, including a Python example and a simple unit test.
      
      Author: zsxwing <zsxwing@gmail.com>
      
      Closes #6955 from zsxwing/kinesis-python and squashes the following commits:
      
      e42e471 [zsxwing] Merge branch 'master' into kinesis-python
      455f7ea [zsxwing] Remove streaming_kinesis_asl_assembly module and simply add the source folder to streaming_kinesis_asl module
      32e6451 [zsxwing] Merge remote-tracking branch 'origin/master' into kinesis-python
      5082d28 [zsxwing] Fix the syntax error for Python 2.6
      fca416b [zsxwing] Fix wrong comparison
      96670ff [zsxwing] Fix the compilation error after merging master
      756a128 [zsxwing] Merge branch 'master' into kinesis-python
      6c37395 [zsxwing] Print stack trace for debug
      7c5cfb0 [zsxwing] RUN_KINESIS_TESTS -> ENABLE_KINESIS_TESTS
      cc9d071 [zsxwing] Fix the python test errors
      466b425 [zsxwing] Add python tests for Kinesis
      e33d505 [zsxwing] Merge remote-tracking branch 'origin/master' into kinesis-python
      3da2601 [zsxwing] Fix the kinesis folder
      687446b [zsxwing] Fix the error message and the maven output path
      add2beb [zsxwing] Merge branch 'master' into kinesis-python
      4957c0b [zsxwing] Add the Python API for Kinesis
      3afc1de8
  9. Jun 12, 2015
    • Tathagata Das's avatar
      [SPARK-7284] [STREAMING] Updated streaming documentation · e9471d34
      Tathagata Das authored
      - Kinesis API updated
      - Kafka version updated, and Python API for Direct Kafka added
      - Added SQLContext.getOrCreate()
      - Added information on how to get partitionId in foreachRDD
      
      Author: Tathagata Das <tathagata.das1565@gmail.com>
      
      Closes #6781 from tdas/SPARK-7284 and squashes the following commits:
      
      aac7be0 [Tathagata Das] Added information on how to get partition id
      a66ec22 [Tathagata Das] Complete the line incomplete line,
      a92ca39 [Tathagata Das] Updated streaming documentation
      e9471d34
  10. Sep 16, 2014
    • Sean Owen's avatar
      SPARK-3069 [DOCS] Build instructions in README are outdated · 61e21fe7
      Sean Owen authored
      Here's my crack at Bertrand's suggestion. The Github `README.md` contains build info that's outdated. It should just point to the current online docs, and reflect that Maven is the primary build now.
      
      (Incidentally, the stanza at the end about contributions of original work should go in https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark too. It won't hurt to be crystal clear about the agreement to license, given that ICLAs are not required of anyone here.)
      
      Author: Sean Owen <sowen@cloudera.com>
      
      Closes #2014 from srowen/SPARK-3069 and squashes the following commits:
      
      501507e [Sean Owen] Note that Zinc is for Maven builds too
      db2bd97 [Sean Owen] sbt -> sbt/sbt and add note about zinc
      be82027 [Sean Owen] Fix additional occurrences of building-with-maven -> building-spark
      91c921f [Sean Owen] Move building-with-maven to building-spark and create a redirect. Update doc links to building-spark.html Add jekyll-redirect-from plugin and make associated config changes (including fixing pygments deprecation). Add example of SBT to README.md
      999544e [Sean Owen] Change "Building Spark with Maven" title to "Building Spark"; reinstate tl;dr info about dev/run-tests in README.md; add brief note about building with SBT
      c18d140 [Sean Owen] Optionally, remove the copy of contributing text from main README.md
      8e83934 [Sean Owen] Add CONTRIBUTING.md to trigger notice on new pull request page
      b1c04a1 [Sean Owen] Refer to current online documentation for building, and remove slightly outdated copy in README.md
      61e21fe7
  11. Sep 06, 2014
    • Tathagata Das's avatar
      [SPARK-2419][Streaming][Docs] More updates to the streaming programming guide · baff7e93
      Tathagata Das authored
      - Improvements to the kinesis integration guide from @cfregly
      - More information about unified input dstreams in main guide
      
      Author: Tathagata Das <tathagata.das1565@gmail.com>
      Author: Chris Fregly <chris@fregly.com>
      
      Closes #2307 from tdas/streaming-doc-fix1 and squashes the following commits:
      
      ec40b5d [Tathagata Das] Updated figure with kinesis
      fdb9c5e [Tathagata Das] Fixed style issues with kinesis guide
      036d219 [Chris Fregly] updated kinesis docs and added an arch diagram
      24f622a [Tathagata Das] More modifications.
      baff7e93
  12. Sep 03, 2014
    • Tathagata Das's avatar
      [SPARK-2419][Streaming][Docs] Updates to the streaming programming guide · a5224079
      Tathagata Das authored
      Updated the main streaming programming guide, and also added source-specific guides for Kafka, Flume, Kinesis.
      
      Author: Tathagata Das <tathagata.das1565@gmail.com>
      Author: Jacek Laskowski <jacek@japila.pl>
      
      Closes #2254 from tdas/streaming-doc-fix and squashes the following commits:
      
      e45c6d7 [Jacek Laskowski] More fixes from an old PR
      5125316 [Tathagata Das] Fixed links
      dc02f26 [Tathagata Das] Refactored streaming kinesis guide and made many other changes.
      acbc3e3 [Tathagata Das] Fixed links between streaming guides.
      cb7007f [Tathagata Das] Added Streaming + Flume integration guide.
      9bd9407 [Tathagata Das] Updated streaming programming guide with additional information from SPARK-2419.
      a5224079
Loading