Skip to content
Snippets Groups Projects
  • Yin Huai's avatar
    5ab9fcfb
    [SPARK-8532] [SQL] In Python's DataFrameWriter,... · 5ab9fcfb
    Yin Huai authored
    [SPARK-8532] [SQL] In Python's DataFrameWriter, save/saveAsTable/json/parquet/jdbc always override mode
    
    https://issues.apache.org/jira/browse/SPARK-8532
    
    This PR has two changes. First, it fixes the bug that save actions (i.e. `save/saveAsTable/json/parquet/jdbc`) always override mode. Second, it adds input argument `partitionBy` to `save/saveAsTable/parquet`.
    
    Author: Yin Huai <yhuai@databricks.com>
    
    Closes #6937 from yhuai/SPARK-8532 and squashes the following commits:
    
    f972d5d [Yin Huai] davies's comment.
    d37abd2 [Yin Huai] style.
    d21290a [Yin Huai] Python doc.
    889eb25 [Yin Huai] Minor refactoring and add partitionBy to save, saveAsTable, and parquet.
    7fbc24b [Yin Huai] Use None instead of "error" as the default value of mode since JVM-side already uses "error" as the default value.
    d696dff [Yin Huai] Python style.
    88eb6c4 [Yin Huai] If mode is "error", do not call mode method.
    c40c461 [Yin Huai] Regression test.
    5ab9fcfb
    History
    [SPARK-8532] [SQL] In Python's DataFrameWriter,...
    Yin Huai authored
    [SPARK-8532] [SQL] In Python's DataFrameWriter, save/saveAsTable/json/parquet/jdbc always override mode
    
    https://issues.apache.org/jira/browse/SPARK-8532
    
    This PR has two changes. First, it fixes the bug that save actions (i.e. `save/saveAsTable/json/parquet/jdbc`) always override mode. Second, it adds input argument `partitionBy` to `save/saveAsTable/parquet`.
    
    Author: Yin Huai <yhuai@databricks.com>
    
    Closes #6937 from yhuai/SPARK-8532 and squashes the following commits:
    
    f972d5d [Yin Huai] davies's comment.
    d37abd2 [Yin Huai] style.
    d21290a [Yin Huai] Python doc.
    889eb25 [Yin Huai] Minor refactoring and add partitionBy to save, saveAsTable, and parquet.
    7fbc24b [Yin Huai] Use None instead of "error" as the default value of mode since JVM-side already uses "error" as the default value.
    d696dff [Yin Huai] Python style.
    88eb6c4 [Yin Huai] If mode is "error", do not call mode method.
    c40c461 [Yin Huai] Regression test.