Skip to content
Blame Permalink
  • Marcelo Vanzin's avatar
    92afaa93
    [SPARK-19307][PYSPARK] Make sure user conf is propagated to SparkContext. · 92afaa93
    Marcelo Vanzin authored 
    The code was failing to propagate the user conf in the case where the
JVM was already initialized, which happens when a user submits a
python script via spark-submit.

Tested with new unit test and by running a python script in a real cluster.

Author: Marcelo Vanzin <vanzin@cloudera.com>

Closes #16682 from vanzin/SPARK-19307.
    92afaa93
    [SPARK-19307][PYSPARK] Make sure user conf is propagated to SparkContext.
    Marcelo Vanzin authored 
    The code was failing to propagate the user conf in the case where the
JVM was already initialized, which happens when a user submits a
python script via spark-submit.

Tested with new unit test and by running a python script in a real cluster.

Author: Marcelo Vanzin <vanzin@cloudera.com>

Closes #16682 from vanzin/SPARK-19307.