Simulating a Spark standalone cluster locally

Add shutdown hook to Executor Runner and execute code to shutdown local cluster in Scheduler Backend

Log cache add/remove messages in block manager.

Added log messages on BlockManagerMaster to reflect block add/remove.
Also did some minor cleanup of storage package code.

Add a limit on the number of parallel fetches in the reduce stage

to the reduce function

one of the cluster schedulers.

issue with shared buffers in the KryoSerializer.

Disable running combiners on map tasks when mergeCombiners function is not specified by the user. 

not specified by the user.

Removed the deserialization cache for ShuffleMapTask

causing concurrency problems (some variables in Shark get set to null).
The cost of task deserialization on slaves is trivial compared with the
execution time of the task anyway.

Cache points in SparkLR example

The broadcast fix is to store values as MEMORY_ONLY_DESER instead of
MEMORY_ONLY, which will save substantial time on serialization.

Broadcast refactoring/cleaning up

Rsync root directory in EC2 script

Size estimator changes for dev

update the results to match the new estimates

oops setting to get deterministic results