-
- Downloads
[SPARK-1981] Add AWS Kinesis streaming support
Author: Chris Fregly <chris@fregly.com> Closes #1434 from cfregly/master and squashes the following commits: 4774581 [Chris Fregly] updated docs, renamed retry to retryRandom to be more clear, removed retries around store() method 0393795 [Chris Fregly] moved Kinesis examples out of examples/ and back into extras/kinesis-asl 691a6be [Chris Fregly] fixed tests and formatting, fixed a bug with JavaKinesisWordCount during union of streams 0e1c67b [Chris Fregly] Merge remote-tracking branch 'upstream/master' 74e5c7c [Chris Fregly] updated per TD's feedback. simplified examples, updated docs e33cbeb [Chris Fregly] Merge remote-tracking branch 'upstream/master' bf614e9 [Chris Fregly] per matei's feedback: moved the kinesis examples into the examples/ dir d17ca6d [Chris Fregly] per TD's feedback: updated docs, simplified the KinesisUtils api 912640c [Chris Fregly] changed the foundKinesis class to be a publically-avail class db3eefd [Chris Fregly] Merge remote-tracking branch 'upstream/master' 21de67f [Chris Fregly] Merge remote-tracking branch 'upstream/master' 6c39561 [Chris Fregly] parameterized the versions of the aws java sdk and kinesis client 338997e [Chris Fregly] improve build docs for kinesis 828f8ae [Chris Fregly] more cleanup e7c8978 [Chris Fregly] Merge remote-tracking branch 'upstream/master' cd68c0d [Chris Fregly] fixed typos and backward compatibility d18e680 [Chris Fregly] Merge remote-tracking branch 'upstream/master' b3b0ff1 [Chris Fregly] [SPARK-1981] Add AWS Kinesis streaming support
Showing
- bin/run-example 2 additions, 1 deletionbin/run-example
- bin/run-example2.cmd 2 additions, 1 deletionbin/run-example2.cmd
- dev/audit-release/audit_release.py 2 additions, 2 deletionsdev/audit-release/audit_release.py
- dev/audit-release/sbt_app_core/src/main/scala/SparkApp.scala 7 additions, 0 deletionsdev/audit-release/sbt_app_core/src/main/scala/SparkApp.scala
- dev/audit-release/sbt_app_kinesis/build.sbt 28 additions, 0 deletionsdev/audit-release/sbt_app_kinesis/build.sbt
- dev/audit-release/sbt_app_kinesis/src/main/scala/SparkApp.scala 33 additions, 0 deletions...dit-release/sbt_app_kinesis/src/main/scala/SparkApp.scala
- dev/create-release/create-release.sh 2 additions, 2 deletionsdev/create-release/create-release.sh
- dev/run-tests 3 additions, 0 deletionsdev/run-tests
- docs/streaming-custom-receivers.md 2 additions, 2 deletionsdocs/streaming-custom-receivers.md
- docs/streaming-kinesis.md 58 additions, 0 deletionsdocs/streaming-kinesis.md
- docs/streaming-programming-guide.md 8 additions, 4 deletionsdocs/streaming-programming-guide.md
- examples/pom.xml 13 additions, 0 deletionsexamples/pom.xml
- extras/kinesis-asl/pom.xml 96 additions, 0 deletionsextras/kinesis-asl/pom.xml
- extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java 180 additions, 0 deletions...che/spark/examples/streaming/JavaKinesisWordCountASL.java
- extras/kinesis-asl/src/main/resources/log4j.properties 37 additions, 0 deletionsextras/kinesis-asl/src/main/resources/log4j.properties
- extras/kinesis-asl/src/main/scala/org/apache/spark/examples/streaming/KinesisWordCountASL.scala 251 additions, 0 deletions...apache/spark/examples/streaming/KinesisWordCountASL.scala
- extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisCheckpointState.scala 56 additions, 0 deletions...ache/spark/streaming/kinesis/KinesisCheckpointState.scala
- extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala 149 additions, 0 deletions.../org/apache/spark/streaming/kinesis/KinesisReceiver.scala
- extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala 212 additions, 0 deletions...ache/spark/streaming/kinesis/KinesisRecordProcessor.scala
- extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisUtils.scala 96 additions, 0 deletions...ala/org/apache/spark/streaming/kinesis/KinesisUtils.scala
Loading
Please register or sign in to comment