Manual merge & cleanup of Shane's Shuffle Performance Optimization

Docs: Mention spark shell's default for MASTER

Master UI has link to Application UI

Conflicts:
	core/src/main/scala/spark/storage/BlockManager.scala
	core/src/test/scala/spark/DistributedSuite.scala
	project/SparkBuild.scala

Conflicts:
	project/SparkBuild.scala

EmptyRDD (master branch 0.8)

Add support for instance local scheduling

SPARK-738: Spark should detect and wrap nonserializable exceptions

1) Add support for HADOOP_CONF_DIR (and/or YARN_CONF_DIR - use either) : which is used to specify the client side configuration directory : which needs to be part of the CLASSPATH.

2) Move from var+=".." to var="$var.." : the former does not work on older bash shells unfortunately.

Actually use the cleaned closure in foreachPartition

dependency on hbase introduces netty-3.2.2 which conflicts with
netty-3.5.3 already in Spark. This caused multiple test failures.

This reverts commit 0f1b7a06, reversing
changes made to aacca1b8.

Fix SPARK-670: EC2 'start' command should require -i option.

hBase example

Conflicts:
	core/src/main/scala/spark/storage/BlockManager.scala
	core/src/main/scala/spark/storage/DiskStore.scala
	project/SparkBuild.scala