Skip to content
Snippets Groups Projects
  • Joseph K. Bradley's avatar
    7e3423b9
    [SPARK-13951][ML][PYTHON] Nested Pipeline persistence · 7e3423b9
    Joseph K. Bradley authored
    Adds support for saving and loading nested ML Pipelines from Python.  Pipeline and PipelineModel do not extend JavaWrapper, but they are able to utilize the JavaMLWriter, JavaMLReader implementations.
    
    Also:
    * Separates out interfaces from Java wrapper implementations for MLWritable, MLReadable, MLWriter, MLReader.
    * Moves methods _stages_java2py, _stages_py2java into Pipeline, PipelineModel as _transfer_stage_from_java, _transfer_stage_to_java
    
    Added new unit test for nested Pipelines.  Abstracted validity check into a helper method for the 2 unit tests.
    
    Author: Joseph K. Bradley <joseph@databricks.com>
    
    Closes #11866 from jkbradley/nested-pipeline-io.
    Closes #11835
    7e3423b9
    History
    [SPARK-13951][ML][PYTHON] Nested Pipeline persistence
    Joseph K. Bradley authored
    Adds support for saving and loading nested ML Pipelines from Python.  Pipeline and PipelineModel do not extend JavaWrapper, but they are able to utilize the JavaMLWriter, JavaMLReader implementations.
    
    Also:
    * Separates out interfaces from Java wrapper implementations for MLWritable, MLReadable, MLWriter, MLReader.
    * Moves methods _stages_java2py, _stages_py2java into Pipeline, PipelineModel as _transfer_stage_from_java, _transfer_stage_to_java
    
    Added new unit test for nested Pipelines.  Abstracted validity check into a helper method for the 2 unit tests.
    
    Author: Joseph K. Bradley <joseph@databricks.com>
    
    Closes #11866 from jkbradley/nested-pipeline-io.
    Closes #11835