Skip to content
Snippets Groups Projects
  • hyukjinkwon's avatar
    99f3c827
    [SPARK-14615][ML][FOLLOWUP] Fix Python examples to use the new ML Vector and... · 99f3c827
    hyukjinkwon authored
    [SPARK-14615][ML][FOLLOWUP] Fix Python examples to use the new ML Vector and Matrix APIs in the ML pipeline based algorithms
    
    ## What changes were proposed in this pull request?
    
    This PR fixes Python examples to use the new ML Vector and Matrix APIs in the ML pipeline based algorithms.
    
    I firstly executed this shell command, `grep -r "from pyspark.mllib" .` and then executed them all.
    Some of tests in `ml` produced the error messages as below:
    
    ```
    pyspark.sql.utils.IllegalArgumentException: u'requirement failed: Input type must be VectorUDT but got org.apache.spark.mllib.linalg.VectorUDTf71b0bce.'
    ```
    
    So, I fixed them to use new ones just identically with some Python tests fixed in https://github.com/apache/spark/pull/12627
    
    ## How was this patch tested?
    
    Manually tested for all the examples listed by `grep -r "from pyspark.mllib" .`.
    
    Author: hyukjinkwon <gurwls223@gmail.com>
    
    Closes #13393 from HyukjinKwon/SPARK-14615.
    99f3c827
    History
    [SPARK-14615][ML][FOLLOWUP] Fix Python examples to use the new ML Vector and...
    hyukjinkwon authored
    [SPARK-14615][ML][FOLLOWUP] Fix Python examples to use the new ML Vector and Matrix APIs in the ML pipeline based algorithms
    
    ## What changes were proposed in this pull request?
    
    This PR fixes Python examples to use the new ML Vector and Matrix APIs in the ML pipeline based algorithms.
    
    I firstly executed this shell command, `grep -r "from pyspark.mllib" .` and then executed them all.
    Some of tests in `ml` produced the error messages as below:
    
    ```
    pyspark.sql.utils.IllegalArgumentException: u'requirement failed: Input type must be VectorUDT but got org.apache.spark.mllib.linalg.VectorUDTf71b0bce.'
    ```
    
    So, I fixed them to use new ones just identically with some Python tests fixed in https://github.com/apache/spark/pull/12627
    
    ## How was this patch tested?
    
    Manually tested for all the examples listed by `grep -r "from pyspark.mllib" .`.
    
    Author: hyukjinkwon <gurwls223@gmail.com>
    
    Closes #13393 from HyukjinKwon/SPARK-14615.