-
- Downloads
[SPARK-8564] [STREAMING] Add the Python API for Kinesis
This PR adds the Python API for Kinesis, including a Python example and a simple unit test. Author: zsxwing <zsxwing@gmail.com> Closes #6955 from zsxwing/kinesis-python and squashes the following commits: e42e471 [zsxwing] Merge branch 'master' into kinesis-python 455f7ea [zsxwing] Remove streaming_kinesis_asl_assembly module and simply add the source folder to streaming_kinesis_asl module 32e6451 [zsxwing] Merge remote-tracking branch 'origin/master' into kinesis-python 5082d28 [zsxwing] Fix the syntax error for Python 2.6 fca416b [zsxwing] Fix wrong comparison 96670ff [zsxwing] Fix the compilation error after merging master 756a128 [zsxwing] Merge branch 'master' into kinesis-python 6c37395 [zsxwing] Print stack trace for debug 7c5cfb0 [zsxwing] RUN_KINESIS_TESTS -> ENABLE_KINESIS_TESTS cc9d071 [zsxwing] Fix the python test errors 466b425 [zsxwing] Add python tests for Kinesis e33d505 [zsxwing] Merge remote-tracking branch 'origin/master' into kinesis-python 3da2601 [zsxwing] Fix the kinesis folder 687446b [zsxwing] Fix the error message and the maven output path add2beb [zsxwing] Merge branch 'master' into kinesis-python 4957c0b [zsxwing] Add the Python API for Kinesis
Showing
- dev/run-tests.py 2 additions, 1 deletiondev/run-tests.py
- dev/sparktestsupport/modules.py 8 additions, 1 deletiondev/sparktestsupport/modules.py
- docs/streaming-kinesis-integration.md 19 additions, 0 deletionsdocs/streaming-kinesis-integration.md
- extras/kinesis-asl-assembly/pom.xml 103 additions, 0 deletionsextras/kinesis-asl-assembly/pom.xml
- extras/kinesis-asl/src/main/python/examples/streaming/kinesis_wordcount_asl.py 81 additions, 0 deletions...c/main/python/examples/streaming/kinesis_wordcount_asl.py
- extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisTestUtils.scala 16 additions, 3 deletions...org/apache/spark/streaming/kinesis/KinesisTestUtils.scala
- extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisUtils.scala 62 additions, 16 deletions...ala/org/apache/spark/streaming/kinesis/KinesisUtils.scala
- pom.xml 1 addition, 0 deletionspom.xml
- project/SparkBuild.scala 3 additions, 3 deletionsproject/SparkBuild.scala
- python/pyspark/streaming/kinesis.py 112 additions, 0 deletionspython/pyspark/streaming/kinesis.py
- python/pyspark/streaming/tests.py 85 additions, 1 deletionpython/pyspark/streaming/tests.py
Loading
Please register or sign in to comment