-
- Downloads
Merge pull request #400 from tdas/dstream-move
Moved DStream and PairDSream to org.apache.spark.streaming.dstream Similar to the package location of `org.apache.spark.rdd.RDD`, `DStream` has been moved from `org.apache.spark.streaming.DStream` to `org.apache.spark.streaming.dstream.DStream`. I know that the package name is a little long, but I think its better to keep it consistent with Spark's structure. Also fixed persistence of windowed DStream. The RDDs generated generated by windowed DStream are essentially unions of underlying RDDs, and persistent these union RDDs would store numerous copies of the underlying data. Instead setting the persistence level on the windowed DStream is made to set the persistence level of the underlying DStream.
No related branches found
No related tags found
Showing
- docs/streaming-programming-guide.md 1 addition, 1 deletiondocs/streaming-programming-guide.md
- examples/src/main/scala/org/apache/spark/streaming/examples/RecoverableNetworkWordCount.scala 1 addition, 1 deletion...park/streaming/examples/RecoverableNetworkWordCount.scala
- external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeUtils.scala 3 additions, 1 deletion...n/scala/org/apache/spark/streaming/flume/FlumeUtils.scala
- external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala 4 additions, 2 deletions...n/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
- external/mqtt/src/main/scala/org/apache/spark/streaming/mqtt/MQTTUtils.scala 3 additions, 1 deletion...ain/scala/org/apache/spark/streaming/mqtt/MQTTUtils.scala
- external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterUtils.scala 6 additions, 1 deletion...ala/org/apache/spark/streaming/twitter/TwitterUtils.scala
- external/zeromq/src/main/scala/org/apache/spark/streaming/zeromq/ZeroMQUtils.scala 2 additions, 1 deletion...scala/org/apache/spark/streaming/zeromq/ZeroMQUtils.scala
- streaming/src/main/scala/org/apache/spark/streaming/DStreamGraph.scala 3 additions, 3 deletions.../main/scala/org/apache/spark/streaming/DStreamGraph.scala
- streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala 3 additions, 2 deletions...n/scala/org/apache/spark/streaming/StreamingContext.scala
- streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaDStream.scala 2 additions, 1 deletion...ala/org/apache/spark/streaming/api/java/JavaDStream.scala
- streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaDStreamLike.scala 1 addition, 0 deletions...org/apache/spark/streaming/api/java/JavaDStreamLike.scala
- streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala 1 addition, 0 deletions...org/apache/spark/streaming/api/java/JavaPairDStream.scala
- streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaStreamingContext.scala 3 additions, 2 deletions...pache/spark/streaming/api/java/JavaStreamingContext.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala 7 additions, 6 deletions...in/scala/org/apache/spark/streaming/dstream/DStream.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/DStreamCheckpointData.scala 4 additions, 6 deletions...pache/spark/streaming/dstream/DStreamCheckpointData.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala 1 addition, 1 deletion...org/apache/spark/streaming/dstream/FileInputDStream.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/FilteredDStream.scala 1 addition, 1 deletion.../org/apache/spark/streaming/dstream/FilteredDStream.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/FlatMapValuedDStream.scala 1 addition, 1 deletion...apache/spark/streaming/dstream/FlatMapValuedDStream.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/FlatMappedDStream.scala 1 addition, 1 deletion...rg/apache/spark/streaming/dstream/FlatMappedDStream.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/ForEachDStream.scala 1 addition, 1 deletion...a/org/apache/spark/streaming/dstream/ForEachDStream.scala
Loading
Please register or sign in to comment