Skip to content
Snippets Groups Projects
  • vundela's avatar
    2f6dd634
    [SPARK-11105] [YARN] Distribute log4j.properties to executors · 2f6dd634
    vundela authored
    Currently log4j.properties file is not uploaded to executor's which is leading them to use the default values. This fix will make sure that file is always uploaded to distributed cache so that executor will use the latest settings.
    
    If user specifies log configurations through --files then executors will be picking configs from --files instead of $SPARK_CONF_DIR/log4j.properties
    
    Author: vundela <vsr@cloudera.com>
    Author: Srinivasa Reddy Vundela <vsr@cloudera.com>
    
    Closes #9118 from vundela/master.
    2f6dd634
    History
    [SPARK-11105] [YARN] Distribute log4j.properties to executors
    vundela authored
    Currently log4j.properties file is not uploaded to executor's which is leading them to use the default values. This fix will make sure that file is always uploaded to distributed cache so that executor will use the latest settings.
    
    If user specifies log configurations through --files then executors will be picking configs from --files instead of $SPARK_CONF_DIR/log4j.properties
    
    Author: vundela <vsr@cloudera.com>
    Author: Srinivasa Reddy Vundela <vsr@cloudera.com>
    
    Closes #9118 from vundela/master.