Skip to content
Snippets Groups Projects
  • Davies Liu's avatar
    872fc669
    [SPARK-4124] [MLlib] [PySpark] simplify serialization in MLlib Python API · 872fc669
    Davies Liu authored
    Create several helper functions to call MLlib Java API, convert the arguments to Java type and convert return value to Python object automatically, this simplify serialization in MLlib Python API very much.
    
    After this, the MLlib Python API does not need to deal with serialization details anymore, it's easier to add new API.
    
    cc mengxr
    
    Author: Davies Liu <davies@databricks.com>
    
    Closes #2995 from davies/cleanup and squashes the following commits:
    
    8fa6ec6 [Davies Liu] address comments
    16b85a0 [Davies Liu] Merge branch 'master' of github.com:apache/spark into cleanup
    43743e5 [Davies Liu] bugfix
    731331f [Davies Liu] simplify serialization in MLlib Python API
    872fc669
    History
    [SPARK-4124] [MLlib] [PySpark] simplify serialization in MLlib Python API
    Davies Liu authored
    Create several helper functions to call MLlib Java API, convert the arguments to Java type and convert return value to Python object automatically, this simplify serialization in MLlib Python API very much.
    
    After this, the MLlib Python API does not need to deal with serialization details anymore, it's easier to add new API.
    
    cc mengxr
    
    Author: Davies Liu <davies@databricks.com>
    
    Closes #2995 from davies/cleanup and squashes the following commits:
    
    8fa6ec6 [Davies Liu] address comments
    16b85a0 [Davies Liu] Merge branch 'master' of github.com:apache/spark into cleanup
    43743e5 [Davies Liu] bugfix
    731331f [Davies Liu] simplify serialization in MLlib Python API