- Nov 10, 2013
-
-
Josh Rosen authored
For now, this only adds MarshalSerializer, but it lays the groundwork for other supporting custom serializers. Many of these mechanisms can also be used to support deserialization of different data formats sent by Java, such as data encoded by MsgPack. This also fixes a bug in SparkContext.union().
-
- Aug 16, 2013
-
-
Andre Schumacher authored
Implementing SPARK-878 for PySpark: adding zip and egg files to context and passing it down to workers which add these to their sys.path
-
- Aug 14, 2013
-
-
Josh Rosen authored
-
- Jul 16, 2013
-
-
Matei Zaharia authored
-
- Jun 21, 2013
-
-
Jey Kottalam authored
-
- Feb 01, 2013
-
-
Josh Rosen authored
The problem was that the gateway was being initialized whenever the pyspark.context module was loaded. The fix uses lazy initialization that occurs only when SparkContext instances are actually constructed. I also made the gateway and jvm variables private. This change results in ~3-4x performance improvement when running the PySpark unit tests.
-
Josh Rosen authored
-
- Jan 25, 2013
-
-
Stephen Haberman authored
-
- Jan 23, 2013
-
-
Josh Rosen authored
Fix minor documentation formatting issues.
-
- Jan 22, 2013
-
-
Josh Rosen authored
-
- Jan 20, 2013
-
-
Josh Rosen authored
-
Josh Rosen authored
-
Josh Rosen authored
-