Skip to content
Snippets Groups Projects
  • hzyaoqin's avatar
    41568e9a
    [SPARK-21637][SPARK-21451][SQL] get `spark.hadoop.*` properties from sysProps to hiveconf · 41568e9a
    hzyaoqin authored
    ## What changes were proposed in this pull request?
    When we use `bin/spark-sql` command configuring `--conf spark.hadoop.foo=bar`, the `SparkSQLCliDriver` initializes an instance of  hiveconf, it does not add `foo->bar` to it.
    this pr gets `spark.hadoop.*` properties from sysProps to this hiveconf
    
    ## How was this patch tested?
    UT
    
    Author: hzyaoqin <hzyaoqin@corp.netease.com>
    Author: Kent Yao <yaooqinn@hotmail.com>
    
    Closes #18668 from yaooqinn/SPARK-21451.
    41568e9a
    History
    [SPARK-21637][SPARK-21451][SQL] get `spark.hadoop.*` properties from sysProps to hiveconf
    hzyaoqin authored
    ## What changes were proposed in this pull request?
    When we use `bin/spark-sql` command configuring `--conf spark.hadoop.foo=bar`, the `SparkSQLCliDriver` initializes an instance of  hiveconf, it does not add `foo->bar` to it.
    this pr gets `spark.hadoop.*` properties from sysProps to this hiveconf
    
    ## How was this patch tested?
    UT
    
    Author: hzyaoqin <hzyaoqin@corp.netease.com>
    Author: Kent Yao <yaooqinn@hotmail.com>
    
    Closes #18668 from yaooqinn/SPARK-21451.