Skip to content
Snippets Groups Projects
  • Devaraj K's avatar
    bc36df12
    [SPARK-13063][YARN] Make the SPARK YARN STAGING DIR as configurable · bc36df12
    Devaraj K authored
    ## What changes were proposed in this pull request?
    Made the SPARK YARN STAGING DIR as configurable with the configuration as 'spark.yarn.staging-dir'.
    
    ## How was this patch tested?
    
    I have verified it manually by running applications on yarn, If the 'spark.yarn.staging-dir' is configured then the value used as staging directory otherwise uses the default value i.e. file system’s home directory for the user.
    
    Author: Devaraj K <devaraj@apache.org>
    
    Closes #12082 from devaraj-kavali/SPARK-13063.
    bc36df12
    History
    [SPARK-13063][YARN] Make the SPARK YARN STAGING DIR as configurable
    Devaraj K authored
    ## What changes were proposed in this pull request?
    Made the SPARK YARN STAGING DIR as configurable with the configuration as 'spark.yarn.staging-dir'.
    
    ## How was this patch tested?
    
    I have verified it manually by running applications on yarn, If the 'spark.yarn.staging-dir' is configured then the value used as staging directory otherwise uses the default value i.e. file system’s home directory for the user.
    
    Author: Devaraj K <devaraj@apache.org>
    
    Closes #12082 from devaraj-kavali/SPARK-13063.