Add shutdown hook to Executor Runner and execute code to shutdown local cluster in Scheduler Backend

Removed the deserialization cache for ShuffleMapTask

causing concurrency problems (some variables in Shark get set to null).
The cost of task deserialization on slaves is trivial compared with the
execution time of the task anyway.

Cache points in SparkLR example

The broadcast fix is to store values as MEMORY_ONLY_DESER instead of
MEMORY_ONLY, which will save substantial time on serialization.

Broadcast refactoring/cleaning up

Rsync root directory in EC2 script

Size estimator changes for dev

update the results to match the new estimates

oops setting to get deterministic results

setting. This is useful to get a deterministic test case

according to architecture and also aligns objects and arrays when computing
instance sizes. Verified using Eclipse Memory Analysis Tool (MAT)

ConnectionManager.

It's still not good because each Future ends up waiting on a lock, but
it seems to work better than Scala Actors, and more importantly it
allows us to use onComplete and other listeners on futures.

Use JavaConversion to get a scala iterator

Detect non-zero exit status from PipedRDD process