Skip to content
Snippets Groups Projects
  • Sean Owen's avatar
    457126e4
    [SPARK-15796][CORE] Reduce spark.memory.fraction default to avoid overrunning... · 457126e4
    Sean Owen authored
    [SPARK-15796][CORE] Reduce spark.memory.fraction default to avoid overrunning old gen in JVM default config
    
    ## What changes were proposed in this pull request?
    
    Reduce `spark.memory.fraction` default to 0.6 in order to make it fit within default JVM old generation size (2/3 heap). See JIRA discussion. This means a full cache doesn't spill into the new gen. CC andrewor14
    
    ## How was this patch tested?
    
    Jenkins tests.
    
    Author: Sean Owen <sowen@cloudera.com>
    
    Closes #13618 from srowen/SPARK-15796.
    457126e4
    History
    [SPARK-15796][CORE] Reduce spark.memory.fraction default to avoid overrunning...
    Sean Owen authored
    [SPARK-15796][CORE] Reduce spark.memory.fraction default to avoid overrunning old gen in JVM default config
    
    ## What changes were proposed in this pull request?
    
    Reduce `spark.memory.fraction` default to 0.6 in order to make it fit within default JVM old generation size (2/3 heap). See JIRA discussion. This means a full cache doesn't spill into the new gen. CC andrewor14
    
    ## How was this patch tested?
    
    Jenkins tests.
    
    Author: Sean Owen <sowen@cloudera.com>
    
    Closes #13618 from srowen/SPARK-15796.