- Feb 21, 2013
-
-
Tathagata Das authored
-
- Feb 20, 2013
-
-
Tathagata Das authored
Conflicts: streaming/src/test/java/spark/streaming/JavaAPISuite.java
-
Tathagata Das authored
-
Tathagata Das authored
-
Tathagata Das authored
Conflicts: core/src/main/scala/spark/rdd/CheckpointRDD.scala streaming/src/main/scala/spark/streaming/dstream/ReducedWindowedDStream.scala
-
Tathagata Das authored
A bug fix to AkkaWordCount example.
-
Prashant Sharma authored
-
- Feb 19, 2013
-
-
Matei Zaharia authored
Don't call System.exit over uncaught exceptions from shutdown hooks
-
Charles Reiss authored
-
Matei Zaharia authored
Fixes a broken link in documentation to issue tracker
-
Matei Zaharia authored
Added a method to create PartitionPruningRDD.
-
Andy Konwinski authored
-
Reynold Xin authored
-
Charles Reiss authored
-
Charles Reiss authored
-
Tathagata Das authored
STREAMING-51: Add RDD type as a type parameter in JavaDStreamLike Edit (streaming/ version)
-
Patrick Wendell authored
-
Patrick Wendell authored
This commit uses the RDD type in `slice`, making it available to both normal and pair RDD's in java. It also updates the signature for `slice` to match changes in the Scala API.
-
Patrick Wendell authored
This is an improved implementation of the `transform` operator in Java. The main difference is that this allows all four possible types of transform functions 1. JavaRDD -> JavaRDD 2. JavaRDD -> JavaPairRDD 3. JavaPairRDD -> JavaPairRDD 4. JavaPairRDD -> JavaRDD whereas previously only (1) and (3) were possible. Conflicts: streaming/src/test/java/spark/streaming/JavaAPISuite.java
-
Patrick Wendell authored
-
Tathagata Das authored
Major modifications to fix driver fault-tolerance with file input stream
-
Tathagata Das authored
-
Tathagata Das authored
-
Tathagata Das authored
Conflicts: docs/plugin-custom-receiver.md streaming/src/main/scala/spark/streaming/StreamingContext.scala streaming/src/main/scala/spark/streaming/dstream/KafkaInputDStream.scala streaming/src/main/scala/spark/streaming/dstream/PluggableInputDStream.scala streaming/src/main/scala/spark/streaming/receivers/ActorReceiver.scala streaming/src/test/scala/spark/streaming/InputStreamsSuite.scala
-
- Feb 18, 2013
-
-
Matei Zaharia authored
Ganglia port change
-
Shivaram Venkataraman authored
-
Shivaram Venkataraman authored
-
Shivaram Venkataraman authored
-
Tathagata Das authored
Changed networkStream to socketStream and pluggableNetworkStream to become networkStream as a way to create streams from arbitrary network receiver.
-
Tathagata Das authored
Conflicts: streaming/src/main/scala/spark/streaming/dstream/KafkaInputDStream.scala streaming/src/main/scala/spark/streaming/dstream/NetworkInputDStream.scala
-
Tathagata Das authored
Added checkpointing and fault-tolerance semantics to the programming guide. Fixed default checkpoint interval to being a multiple of slide duration. Fixed visibility of some classes and objects to clean up docs.
-
Matei Zaharia authored
-
Matei Zaharia authored
-
- Feb 17, 2013
-
-
Matei Zaharia authored
-
Tathagata Das authored
recovering from 1st failure - Made the scheduler to checkpoint after clearing old metadata which ensures that a new checkpoint is written as soon as at least one batch gets computed while recovering from a failure. This ensures that if there is a 2nd failure while recovering from 1st failure, the system start 2nd recovery from a newer checkpoint. - Modified Checkpoint writer to write checkpoint in a different thread. - Added a check to make sure that compute for InputDStreams gets called only for strictly increasing times. - Changed implementation of slice to call getOrCompute on parent DStream in time-increasing order. - Added testcase to test slice. - Fixed testGroupByKeyAndWindow testcase in JavaAPISuite to verify results with expected output in an order-independent manner.
-
Matei Zaharia authored
-
Matei Zaharia authored
Switch spark_ec2.py to use the new spark-ec2 scripts.
-
- Feb 16, 2013
-
-
Matei Zaharia authored
Move ParallelCollection into spark.rdd package.
-
Matei Zaharia authored
Make CoGroupedRDDs explicitly have the same key type.
-
Stephen Haberman authored
-