a. Fix the port number by reading it from the bound channel
b. Fix the shutdown sequence to make sure we actually block on the channel
c. Fix the unit test to use two JVMs.

Fix CheckpointRDD java.io.FileNotFoundException when calling getPreferredLocations

Use ec2-metadata in start-slave.sh to detect if running on EC2

[SPARK-663]Implement Fair Scheduler in Spark Cluster Scheduler 

NonJavaSerializableClass should not be Java serializable...

serializable.

Automatically configure Netty port.

local-cluster pass. Previously they were failing because Netty was
trying to bind to the same port for all processes.

Pair programmed with @shivaram.

bug fix: Shuffle block iterator is ignoring the shuffle serializer setting.

block iterators for local blocks. Also added a unit test for that.

PR #419 applied the same change, but only to start-master.sh,
so some workers were still starting their web UI's using internal
addresses.

This should finally fix SPARK-613.

Exclude old versions of Netty from Maven-based build

Use ARRAY_SAMPLE_SIZE constant instead of hard-coded 100.0 in SizeEstimator

DistributedSuite: remove dead code

Throw exception if TaskResult exceeds Akka frame size

called.

Conflicts:
	core/src/main/scala/spark/SparkContext.scala
	core/src/main/scala/spark/scheduler/cluster/ClusterScheduler.scala
	core/src/main/scala/spark/scheduler/cluster/TaskSetManager.scala

Maven build fix & two other small changes

check for core classes in run. This fixed the problem that core tests
depend on whether repl module is compiled or not.

Manual merge & cleanup of Shane's Shuffle Performance Optimization