Skip to content
Snippets Groups Projects
Commit 809c785b authored by Davies Liu's avatar Davies Liu Committed by Xiangrui Meng
Browse files

[SPARK-2652] [PySpark] donot use KyroSerializer as default serializer

KyroSerializer can not serialize customized class without registered explicitly, use it as default serializer in PySpark will introduce some regression in MLlib.

cc mengxr

Author: Davies Liu <davies@databricks.com>

Closes #2916 from davies/revert and squashes the following commits:

43eb6d3 [Davies Liu] donot use KyroSerializer as default serializer
parent 0aea2289
No related branches found
No related tags found
No related merge requests found
...@@ -43,7 +43,6 @@ __all__ = ['SparkContext'] ...@@ -43,7 +43,6 @@ __all__ = ['SparkContext']
# These are special default configs for PySpark, they will overwrite # These are special default configs for PySpark, they will overwrite
# the default ones for Spark if they are not configured by user. # the default ones for Spark if they are not configured by user.
DEFAULT_CONFIGS = { DEFAULT_CONFIGS = {
"spark.serializer": "org.apache.spark.serializer.KryoSerializer",
"spark.serializer.objectStreamReset": 100, "spark.serializer.objectStreamReset": 100,
"spark.rdd.compress": True, "spark.rdd.compress": True,
} }
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment