-
- Downloads
Remove Pickle-wrapping of Java objects in PySpark.
If we support custom serializers, the Python worker will know what type of input to expect, so we won't need to wrap Tuple2 and Strings into pickled tuples and strings.
Showing
- core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala 39 additions, 67 deletions...rc/main/scala/org/apache/spark/api/python/PythonRDD.scala
- python/pyspark/context.py 5 additions, 5 deletionspython/pyspark/context.py
- python/pyspark/rdd.py 7 additions, 4 deletionspython/pyspark/rdd.py
- python/pyspark/serializers.py 18 additions, 0 deletionspython/pyspark/serializers.py
- python/pyspark/worker.py 9 additions, 5 deletionspython/pyspark/worker.py
Loading
Please register or sign in to comment