Skip to content
Snippets Groups Projects
  • Yong Tang's avatar
    7db56244
    [SPARK-14368][PYSPARK] Support python.spark.worker.memory with upper-case unit. · 7db56244
    Yong Tang authored
    ## What changes were proposed in this pull request?
    
    This fix tries to address the issue in PySpark where `spark.python.worker.memory`
    could only be configured with a lower case unit (`k`, `m`, `g`, `t`). This fix
    allows the upper case unit (`K`, `M`, `G`, `T`) to be used as well. This is to
    conform to the JVM memory string as is specified in the documentation .
    
    ## How was this patch tested?
    
    This fix adds additional test to cover the changes.
    
    Author: Yong Tang <yong.tang.github@outlook.com>
    
    Closes #12163 from yongtang/SPARK-14368.
    7db56244
    History
    [SPARK-14368][PYSPARK] Support python.spark.worker.memory with upper-case unit.
    Yong Tang authored
    ## What changes were proposed in this pull request?
    
    This fix tries to address the issue in PySpark where `spark.python.worker.memory`
    could only be configured with a lower case unit (`k`, `m`, `g`, `t`). This fix
    allows the upper case unit (`K`, `M`, `G`, `T`) to be used as well. This is to
    conform to the JVM memory string as is specified in the documentation .
    
    ## How was this patch tested?
    
    This fix adds additional test to cover the changes.
    
    Author: Yong Tang <yong.tang.github@outlook.com>
    
    Closes #12163 from yongtang/SPARK-14368.