-
- Downloads
Merge pull request #416 from tdas/filestream-fix
Removed unnecessary DStream operations and updated docs Removed StreamingContext.registerInputStream and registerOutputStream - they were useless. InputDStream has been made to register itself, and just registering a DStream as output stream cause RDD objects to be created but the RDDs will not be computed at all.. Also made DStream.register() private[streaming] for the same reasons. Updated docs, specially added package documentation for streaming package. Also, changed NetworkWordCount's input storage level to use MEMORY_ONLY, replication on the local machine causes warning messages (as replication fails) which is scary for a new user trying out his/her first example.
No related branches found
No related tags found
Showing
- docs/streaming-programming-guide.md 17 additions, 15 deletionsdocs/streaming-programming-guide.md
- examples/src/main/scala/org/apache/spark/streaming/examples/NetworkWordCount.scala 2 additions, 1 deletion...rg/apache/spark/streaming/examples/NetworkWordCount.scala
- external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeUtils.scala 0 additions, 1 deletion...n/scala/org/apache/spark/streaming/flume/FlumeUtils.scala
- external/flume/src/test/scala/org/apache/spark/streaming/flume/FlumeStreamSuite.scala 1 addition, 1 deletion...a/org/apache/spark/streaming/flume/FlumeStreamSuite.scala
- external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala 1 addition, 3 deletions...n/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
- external/mqtt/src/main/scala/org/apache/spark/streaming/mqtt/MQTTUtils.scala 1 addition, 3 deletions...ain/scala/org/apache/spark/streaming/mqtt/MQTTUtils.scala
- external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterUtils.scala 1 addition, 3 deletions...ala/org/apache/spark/streaming/twitter/TwitterUtils.scala
- streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala 6 additions, 34 deletions...n/scala/org/apache/spark/streaming/StreamingContext.scala
- streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaStreamingContext.scala 0 additions, 7 deletions...pache/spark/streaming/api/java/JavaStreamingContext.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala 10 additions, 10 deletions...in/scala/org/apache/spark/streaming/dstream/DStream.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/InputDStream.scala 14 additions, 10 deletions...ala/org/apache/spark/streaming/dstream/InputDStream.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala 5 additions, 4 deletions.../apache/spark/streaming/dstream/NetworkInputDStream.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/PairDStreamFunctions.scala 5 additions, 0 deletions...apache/spark/streaming/dstream/PairDStreamFunctions.scala
- streaming/src/main/scala/org/apache/spark/streaming/package.scala 38 additions, 0 deletions...g/src/main/scala/org/apache/spark/streaming/package.scala
- streaming/src/main/scala/org/apache/spark/streaming/util/MasterFailureTest.scala 1 addition, 1 deletion...a/org/apache/spark/streaming/util/MasterFailureTest.scala
- streaming/src/test/java/org/apache/spark/streaming/JavaTestUtils.scala 1 addition, 2 deletions.../test/java/org/apache/spark/streaming/JavaTestUtils.scala
- streaming/src/test/scala/org/apache/spark/streaming/BasicOperationsSuite.scala 0 additions, 1 deletion...ala/org/apache/spark/streaming/BasicOperationsSuite.scala
- streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala 1 addition, 1 deletion...st/scala/org/apache/spark/streaming/CheckpointSuite.scala
- streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala 4 additions, 4 deletions.../scala/org/apache/spark/streaming/InputStreamsSuite.scala
- streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala 0 additions, 1 deletion...la/org/apache/spark/streaming/StreamingContextSuite.scala
Loading
Please register or sign in to comment