fixes various bugs in the previous code when running on Mesos.

across projects).

Fixed nasty corner case bug in ByteBufferInputStream. Could not add a test case for this as I could not figure out how to deterministically reproduce the bug in a short testcase.

ConnectionManager and DAGScheduler properly, plus a fix to
LocalScheduler that was not merged in from 0.5 and was actually caught
by one of the tests.

broadcast. Fixes #139.

RDD partitioning in more cases where it's possible, and use iterators
instead of materializing collections when doing joins.

jointly with Tathagata Das and Haoyuan Li. This commit imports the changes
and ports them to Mesos 0.9, but does not yet pass unit tests due to
various classes not supporting a graceful stop() yet.

cluster instead of on the master. Fixes #110.

SizeEstimator. Also added a test for SizeEstimator. Fixes #136.

to be serialized in closure serialization. This can significantly reduce
the task setup time in Shark when the query involves a large number of
(Hive) partitions.

it was pretty ugly that put() should return -1 for failures.