Skip to content
  • Erik Selin's avatar
    8edc9d03
    [SPARK-1468] Modify the partition function used by partitionBy. · 8edc9d03
    Erik Selin authored
    Make partitionBy use a tweaked version of hash as its default partition function
    since the python hash function does not consistently assign the same value
    to None across python processes.
    
    Associated JIRA at https://issues.apache.org/jira/browse/SPARK-1468
    
    Author: Erik Selin <erik.selin@jadedpixel.com>
    
    Closes #371 from tyro89/consistent_hashing and squashes the following commits:
    
    201c301 [Erik Selin] Make partitionBy use a tweaked version of hash as its default partition function since the python hash function does not consistently assign the same value to None across python processes.
    8edc9d03
    [SPARK-1468] Modify the partition function used by partitionBy.
    Erik Selin authored
    Make partitionBy use a tweaked version of hash as its default partition function
    since the python hash function does not consistently assign the same value
    to None across python processes.
    
    Associated JIRA at https://issues.apache.org/jira/browse/SPARK-1468
    
    Author: Erik Selin <erik.selin@jadedpixel.com>
    
    Closes #371 from tyro89/consistent_hashing and squashes the following commits:
    
    201c301 [Erik Selin] Make partitionBy use a tweaked version of hash as its default partition function since the python hash function does not consistently assign the same value to None across python processes.
Loading