Skip to content
Snippets Groups Projects
  • Zhe Sun's avatar
    accbed7c
    [SPARK-19797][DOC] ML pipeline document correction · accbed7c
    Zhe Sun authored
    ## What changes were proposed in this pull request?
    Description about pipeline in this paragraph is incorrect https://spark.apache.org/docs/latest/ml-pipeline.html#how-it-works
    
    
    
    > If the Pipeline had more **stages**, it would call the LogisticRegressionModel’s transform() method on the DataFrame before passing the DataFrame to the next stage.
    
    Reason: Transformer could also be a stage. But only another Estimator will invoke an transform call and pass the data to next stage. The description in the document misleads ML pipeline users.
    
    ## How was this patch tested?
    This is a tiny modification of **docs/ml-pipelines.md**. I jekyll build the modification and check the compiled document.
    
    Author: Zhe Sun <ymwdalex@gmail.com>
    
    Closes #17137 from ymwdalex/SPARK-19797-ML-pipeline-document-correction.
    
    (cherry picked from commit 0bac3e4c)
    Signed-off-by: default avatarSean Owen <sowen@cloudera.com>
    accbed7c
    History
    [SPARK-19797][DOC] ML pipeline document correction
    Zhe Sun authored
    ## What changes were proposed in this pull request?
    Description about pipeline in this paragraph is incorrect https://spark.apache.org/docs/latest/ml-pipeline.html#how-it-works
    
    
    
    > If the Pipeline had more **stages**, it would call the LogisticRegressionModel’s transform() method on the DataFrame before passing the DataFrame to the next stage.
    
    Reason: Transformer could also be a stage. But only another Estimator will invoke an transform call and pass the data to next stage. The description in the document misleads ML pipeline users.
    
    ## How was this patch tested?
    This is a tiny modification of **docs/ml-pipelines.md**. I jekyll build the modification and check the compiled document.
    
    Author: Zhe Sun <ymwdalex@gmail.com>
    
    Closes #17137 from ymwdalex/SPARK-19797-ML-pipeline-document-correction.
    
    (cherry picked from commit 0bac3e4c)
    Signed-off-by: default avatarSean Owen <sowen@cloudera.com>