Skip to content
Snippets Groups Projects
  • Evan Sparks's avatar
    6ed7e2cd
    Use numpy directly for matrix multiply. · 6ed7e2cd
    Evan Sparks authored
    Using matrix multiply to compute XtX and XtY yields a 5-20x speedup depending on problem size.
    
    For example - the following takes 19s locally after this change vs. 5m21s before the change. (16x speedup).
    bin/pyspark examples/src/main/python/als.py local[8] 1000 1000 50 10 10
    
    Author: Evan Sparks <evan.sparks@gmail.com>
    
    Closes #687 from etrain/patch-1 and squashes the following commits:
    
    e094dbc [Evan Sparks] Touching only diaganols on update.
    d1ab9b6 [Evan Sparks] Use numpy directly for matrix multiply.
    6ed7e2cd
    History
    Use numpy directly for matrix multiply.
    Evan Sparks authored
    Using matrix multiply to compute XtX and XtY yields a 5-20x speedup depending on problem size.
    
    For example - the following takes 19s locally after this change vs. 5m21s before the change. (16x speedup).
    bin/pyspark examples/src/main/python/als.py local[8] 1000 1000 50 10 10
    
    Author: Evan Sparks <evan.sparks@gmail.com>
    
    Closes #687 from etrain/patch-1 and squashes the following commits:
    
    e094dbc [Evan Sparks] Touching only diaganols on update.
    d1ab9b6 [Evan Sparks] Use numpy directly for matrix multiply.