RDD partitioning in more cases where it's possible, and use iterators
instead of materializing collections when doing joins.

cluster instead of on the master. Fixes #110.

SizeEstimator. Also added a test for SizeEstimator. Fixes #136.

of a weird file dediated to accessing this variable).

Little refactoring and unit tests for CacheTrackerActor

BoundedMemoryCache.put should fail when estimated size of 'value' is larger than cache capacity

Return size estimation, cache usage, and cache capacity from slave nodes to CacheTracker

to be serialized in closure serialization. This can significantly reduce
the task setup time in Shark when the query involves a large number of
(Hive) partitions.

it was pretty ugly that put() should return -1 for failures.

trackor. This is essential for building a dashboard to see the status of
caches on all slaves.

from the same dataset.

Force serialize/deserialize task results in local execution mode.

End task instead of just exiting in LocalScheduler for tasks that throw exceptions

would throw an exception for local tasks).

Added the ability to set environmental variables in piped rdd.

Added an option (spark.closure.serializer) to specify the serializer for closures.

tasks.

closures. This enables using Kryo as the closure serializer.

Report entry dropping in BoundedMemoryCache

BoundedMemoryCache, in case the JVM was not started with its initial
heap size equaling its maximum one (-Xms == -Xmx).