Skip to content
  • Michelangelo D'Agostino's avatar
    1a623b2e
    SPARK-3770: Make userFeatures accessible from python · 1a623b2e
    Michelangelo D'Agostino authored
    https://issues.apache.org/jira/browse/SPARK-3770
    
    We need access to the underlying latent user features from python. However, the userFeatures RDD from the MatrixFactorizationModel isn't accessible from the python bindings. I've added a method to the underlying scala class to turn the RDD[(Int, Array[Double])] to an RDD[String]. This is then accessed from the python recommendation.py
    
    Author: Michelangelo D'Agostino <mdagostino@civisanalytics.com>
    
    Closes #2636 from mdagost/mf_user_features and squashes the following commits:
    
    c98f9e2 [Michelangelo D'Agostino] Added unit tests for userFeatures and productFeatures and merged master.
    d5eadf8 [Michelangelo D'Agostino] Merge branch 'master' into mf_user_features
    2481a2a [Michelangelo D'Agostino] Merged master and resolved conflict.
    a6ffb96 [Michelangelo D'Agostino] Eliminated a function from our first approach to this problem that is no longer needed now that we added the fromTuple2RDD function.
    2aa1bf8 [Michelangelo D'Agostino] Implemented a function called fromTuple2RDD in PythonMLLibAPI and used it to expose the MF userFeatures and productFeatures in python.
    34cb2a2 [Michelangelo D'Agostino] A couple of lint cleanups and a comment.
    cdd98e3 [Michelangelo D'Agostino] It's working now.
    e1fbe5e [Michelangelo D'Agostino] Added scala function to stringify userFeatures for access in python.
    1a623b2e
    SPARK-3770: Make userFeatures accessible from python
    Michelangelo D'Agostino authored
    https://issues.apache.org/jira/browse/SPARK-3770
    
    We need access to the underlying latent user features from python. However, the userFeatures RDD from the MatrixFactorizationModel isn't accessible from the python bindings. I've added a method to the underlying scala class to turn the RDD[(Int, Array[Double])] to an RDD[String]. This is then accessed from the python recommendation.py
    
    Author: Michelangelo D'Agostino <mdagostino@civisanalytics.com>
    
    Closes #2636 from mdagost/mf_user_features and squashes the following commits:
    
    c98f9e2 [Michelangelo D'Agostino] Added unit tests for userFeatures and productFeatures and merged master.
    d5eadf8 [Michelangelo D'Agostino] Merge branch 'master' into mf_user_features
    2481a2a [Michelangelo D'Agostino] Merged master and resolved conflict.
    a6ffb96 [Michelangelo D'Agostino] Eliminated a function from our first approach to this problem that is no longer needed now that we added the fromTuple2RDD function.
    2aa1bf8 [Michelangelo D'Agostino] Implemented a function called fromTuple2RDD in PythonMLLibAPI and used it to expose the MF userFeatures and productFeatures in python.
    34cb2a2 [Michelangelo D'Agostino] A couple of lint cleanups and a comment.
    cdd98e3 [Michelangelo D'Agostino] It's working now.
    e1fbe5e [Michelangelo D'Agostino] Added scala function to stringify userFeatures for access in python.
Loading