Do not launch JavaGateways on workers (SPARK-674).

The problem was that the gateway was being initialized whenever the
pyspark.context module was loaded.  The fix uses lazy initialization
that occurs only when SparkContext instances are actually constructed.

I also made the gateway and jvm variables private.

This change results in ~3-4x performance improvement when running the
PySpark unit tests.

Changed PartitionPruningRDD's split to make sure it returns the correct split index.

Fix stdout redirection in PySpark.

SPARK-673: Capture and re-throw Python exceptions

Remove activation of profiles by default

This patch alters the Python <-> executor protocol to pass on
exception data when they occur in user Python code.

split index.

See the discussion at https://github.com/mesos/spark/pull/355 for why
default profile activation is a problem.

Minor improvements to PySpark docs

Also, adds a line in doc explaining how to use.

It's nicer if all the commands you need are made explicit.

Remember ConnectionManagerId used to initiate SendingConnections

Make ExecutorIDs include SlaveIDs when running Mesos

Include message and exitStatus if availalbe.

the Mesos ExecutorID as a Spark ExecutorID.

This prevents ConnectionManager from getting confused if a machine
has multiple host names and the one getHostName() finds happens
not to be the one that was passed from, e.g., the BlockManagerMaster.

Conflicts:
	core/src/main/scala/spark/deploy/master/Master.scala

Add RDD.toDebugString.

Replace old 'master' term with 'driver'.

- RDD's getDependencies and getSplits methods are now guaranteed to be
  called only once, so subclasses can safely do computation in there
  without worrying about caching the results.

- The management of a "splits_" variable that is cleared out when we
  checkpoint an RDD is now done in the RDD class.

- A few of the RDD subclasses are simpler.

- CheckpointRDD's compute() method no longer assumes that it is given a
  CheckpointRDDSplit -- it can work just as well on a split from the
  original RDD, because it only looks at its index. This is important
  because things like UnionRDD and ZippedRDD remember the parent's
  splits as part of their own and wouldn't work on checkpointed parents.

- RDD.iterator can now reuse cached data if an RDD is computed before it
  is checkpointed. It seems like it wouldn't do this before (it always
  called iterator() on the CheckpointRDD, which read from HDFS).

SPARK-658: Adding logging of stage duration

Original idea by Nathan Kronenfeld.

Conflicts:
	core/src/main/scala/spark/SparkContext.scala
	core/src/main/scala/spark/SparkEnv.scala
	core/src/main/scala/spark/deploy/LocalSparkCluster.scala
	core/src/main/scala/spark/executor/StandaloneExecutorBackend.scala
	core/src/main/scala/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala
	core/src/main/scala/spark/scheduler/cluster/StandaloneClusterMessage.scala
	core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala
	core/src/main/scala/spark/storage/BlockManagerMaster.scala
	core/src/main/scala/spark/storage/ThreadingTest.scala
	core/src/test/scala/spark/MapOutputTrackerSuite.scala

Some DEBUG-level log cleanup.

add long and float accumulatorparams